Automate Tests For Multiple Drupal Sites Using A Single Script

This article is for those who want to implement Behavior-Driven Development (BDD) using Behat in their Drupal projects in general. It is also helpful for those who are working on a collection of similar sites like a multi-site setup or an Acquia Cloud Site Factory setup where sites exhibit similar behavior.

Before you begin, you’ll need to have Behat set up for Drupal. Check out Shweta’s articles on Automated Testing With Behat: Part 1 and Automated Testing With Behat: Part 2. 

Here’s what we’ll cover. 

• How to write non-JavaScript enabled tests using the Goutte driver.
• How to write JavaScript enabled tests using the Selenium driver.
• How to configure behat.yml for multiple sites.
• How to configure and create a PHP script to feed multiple URLs to Behat.
• How to generate corresponding reports for each site the test runs on.
   

Step 1

Create a composer.json file in the root directory and paste the extensions you want to install. Here is a reference composer.json file.

Open a terminal, navigate to the project’s root directory and paste the following command:

composer install 

Step 2

Feature file

Every *.feature file formally consists of a single feature. Lines with the keyword Feature:  followed by three organized lines start a feature.

A feature normally includes a list of scenarios. User can write until the first scenario, which starts with Scenario:  on a new line. User can apply tags to combination features and scenarios together.

Every scenario consists of a record of steps, which must begin with one of the keywords Given, When, Then, But or And. Behat employs them all equally.

Take a look at the feature file below:

Reference Scenarios: 

Feature file 1 

Feature file 2 

Goutte Vs Selenium 2 

GoutteDriver

There are several commonly used browser libraries that extend Behat functionalities. Some, like Goutte, are very fast. This is a headless browser. GoutteDriver works like a bridge between Behat and the browser, handling all the interactions between them. It offers some advantages, such as it’s easy to set up, and test execution is fast. However, it does not support JavaScript (JS) and AJAX related calls.

Below GoutteDriver is a pure PHP library available through Composer.

$ composer require behat/mink-goutte-driver

Selenium2Driver

Selenium2Driver is mostly used in JS calls. This is the beauty of using Mink—you can get the best of these drivers just by configuring them in your YML file. 

When working on a web app, not considering JS and AJAX would be unrealistic. There would be many scenarios where users need to wait for JavaScript to be loaded entirely, or wait till the AJAX request is completely processed, or the script might fail due to element visibility issues. There are various drivers offered by Mink which can help us overcome this problem, but the Selenium2 driver assists users in fulfilling user requirements.

We just need to configure Selenium2Driver in our behat.yml file, and the user is supposed to tag their feature as @javascript, which tells the Behat runner to use Selenium2Driver instead of the Goutte driver. 

Selenium2Driver is available through Composer:

$ composer require behat/mink-selenium2-driver

If you are considering Goutte Vs Selenium 2, you can see a feature comparison in the screenshot below:

JavaScript Vs Non-JavaScript based scenarios

Non-javascript based scenarios are simple. We don’t need to define any tag and it will work accordingly.

For JavaScript based scenarios, we need to tell Behat and Mink to run this scenario in a different session (with a different browser emulator). Mink comes with a special hook, that searches @javascript.

Take a look at this reference screenshot:

The reason behind this implementation

When manual testing, if a QA engineer needs to test more than 300 sites with similar scenarios, that would be a hectic task involving a lot of time and resources. To make this task easier, we’ve found a solution to test multiple Drupal sites with single script execution. 

There are a few reasons behind this implementation:

• Drupal Core update.
• Helpful to test more than 300 sites
• Manual QA can be a complex task in terms of the resources and number of days involved
   

Solution

We will list all the sites in a single file and have a script loop over it.

1. Excel/sites.csv
   
   Create a file called “sites.csv” and place it in the root directory. In this file, you are supposed to add the site URLs and report unique name.
   
   BehatSite1,dev-behatsite1.pantheonsite.iobehatSite1 - HTML file name. When the report is generated, this filename will be used.
   
   Dev-behatsite1.pantheonsite.io - Website URL name
    
2. Feed URLs to PHP script
   
   It read the URLs from CSV file one by one via php script. 
3. PHP script iterates through the list
   
   Set the loop and pick URLs one by one and test accordingly.
    
4. Run Behat per site and generate reports
   
   Execute the Behat script with command “Runbehat.php” and it will generate the HTML reports.
    

Here is a reference video recording for script execution: 

Extension used to generate structured HTML reports

Here is the extension: “emuse/behat-html-formatter”

Command: composer require emuse/behat-html-formatter

This will create an HTML formatter for Behat.

Here is a reference image for reports structure:

Behat.yml Configuration

Here is the behat.yml configuration, where you will notice that we didn’t define “Base_url” because it will automatically be detected in behat.yml.

The simple yet powerful Script that runs our tests

Here is the “Runbehat.php” file which is the main source. The functions have been written for single script execution.

Command: PHP runbehat.php

You can see the code here.

Reports

With “emuse/behat-html-formatter”, extension reports will be generated. Take a look at the example report screenshot below.