Headless Execution of WebDriver Tests- Chrome Browser

WebDriver
Share
Headless Execution WebDriver Tests- Chrome

In the WebDriver Series, you can read lots of advanced tips and tricks about automated testing with WebDriver. In today's publication, I am going to share with you how to speed up your tests' execution through the usage of the newest Headless Execution mode of Google Chrome. Moreover, I made a couple of benchmarks to compare its speed to the rest of the major browsers.

Test Case

I have created a particular HTML page that we are going to use in our tests. There I have put 100 of each of six different HTML elements- text, select, submit, color, data, and radio. Below you can find the page without the repetition of the elements.

<!DOCTYPE html>
<html lang="en" xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8" />
<title></title>
</head>
<body>
<input type="text" name="fname"><br>
<select>
<option value="volvo">Volvo</option>
<option value="saab">Saab</option>
<option value="mercedes">Mercedes</option>
<option value="audi">Audi</option>
</select><br>
<input type="submit" value="Submit"><br>
<input type="color" value="#ff0000"><br>
<input type="date" value="2017-06-01" min="1980-04-01" max="2099-04-30"><br>
<input type="radio" name="gender" value="male"> Male<br>
</body>
</html>
Test HTML Page

You can find the full test page on the following URL.

Headless Execution Chrome Driver

It is quite simple to run your tests in the headless mode. You need simply to add the "headless" argument to the ChromeOptions object.

var options = new ChromeOptions();
options.AddArguments("headless");
using (IWebDriver driver = new ChromeDriver(options))
{
// the rest of your test
}

Benchmark against Other Browsers

Test Case

private void PerformTestOperations(IWebDriver driver)
{
string testPagePath =
"http://htmlpreview.github.io/?https://github.com/angelovstanton/AutomateThePlanet/blob/master/WebDriver-Series/TestPage.html";
driver.Navigate().GoToUrl(testPagePath);
driver.Manage().Timeouts().PageLoad = TimeSpan.FromSeconds(10);
driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(10);
var textBoxes = driver.FindElements(By.Name("fname"));
foreach (var textBox in textBoxes)
{
textBox.SendKeys(Guid.NewGuid().ToString());
}
var selects = driver.FindElements(By.TagName("select"));
foreach (var select in selects)
{
var selectElement = new SelectElement(select);
selectElement.SelectByText("Mercedes");
}
var submits = driver.FindElements(By.XPath("//input[@type='submit']"));
foreach (var submit in submits)
{
submit.Click();
}
var colors = driver.FindElements(By.XPath("//input[@type='color']"));
foreach (var color in colors)
{
SetValueAttribute(driver, color, "#000000");
}
var dates = driver.FindElements(By.XPath("//input[@type='date']"));
foreach (var date in dates)
{
SetValueAttribute(driver, date, "2020-06-01");
}
var radioButtons = driver.FindElements(By.XPath("//input[@type='radio']"));
foreach (var radio in radioButtons)
{
radio.Click();
}
}
private void SetValueAttribute(IWebDriver driver, IWebElement element, string value)
{
SetAttribute(driver, element, "value", value);
}
private void SetAttribute(IWebDriver driver, IWebElement element, string attributeName, string attributeValue)
{
driver.ExecuteJavaScript
(
"arguments[0].setAttribute(arguments[1], arguments[2]);",
element,
attributeName,
attributeValue);
}

First, we open the page that I previously showed you- the one with over 600 HTML elements. Then we set some timeouts. We find all elements of each type using different locators. For the HTML 5 elements (date and color) we use a little bit of JavaScript code to set the value of the elements. For the rest of the items, we use the native WebDriver methods.

Benchmark Execution Time

To make proper conclusions about the speed of the different browsers, we will use the method Profile. It will execute the above test a specific number of times (in our case 10 times) and measure the total and average test execution times.

private void Profile(string testResultsFileName, int iterations, Action actionToProfile)
{
var desktopPath = Environment.GetFolderPath(Environment.SpecialFolder.Desktop);
var resultsPath = Path.Combine(desktopPath, "BenchmarkTestResults",
string.Concat(testResultsFileName, "_", Guid.NewGuid().ToString(), ".txt"));
var writer = new StreamWriter(resultsPath);
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
var watch = new Stopwatch();
watch.Start();
for (var i = 0; i < iterations; i++)
{
actionToProfile();
}
watch.Stop();
writer.WriteLine("Total: {0:0.00} ms ({1:N0} ticks) (over {2:N0} iterations)",
watch.ElapsedMilliseconds, watch.ElapsedTicks, iterations);
var avgElapsedMillisecondsPerRun = watch.ElapsedMilliseconds / iterations;
var avgElapsedTicksPerRun = watch.ElapsedMilliseconds / iterations;
writer.WriteLine("AVG: {0:0.00} ms ({1:N0} ticks) (over {2:N0} iterations)",
avgElapsedMillisecondsPerRun, avgElapsedTicksPerRun, iterations);
writer.Flush();
writer.Close();
}

Benchmark Tests

[Test]
public void TestChromeExecutionTime()
{
Profile
(
"TestChromeExecutionTime",
10,
() =>
{
using (IWebDriver driver = new ChromeDriver())
{
PerformTestOperations(driver);
}
}
);
}
[Test]
public void TestFirefoxExecutionTime()
{
Profile
(
"TestFirefoxExecutionTime",
10,
() =>
{
using (IWebDriver driver = new FirefoxDriver())
{
PerformTestOperations(driver);
}
}
);
}
[Test]
public void TestEdgeExecutionTime()
{
Profile
(
"TestEdgeExecutionTime",
10,
() =>
{
using (IWebDriver driver = new EdgeDriver())
{
PerformTestOperations(driver);
}
}
);
}
[Test]
public void TestPhantomJsExecutionTime()
{
Profile
(
"TestPhantomJsExecutionTime",
10,
() =>
{
using (IWebDriver driver = new PhantomJSDriver())
{
PerformTestOperations(driver);
}
}
);
}
[Test]
public void TestChromeHeadlessExecutionTime()
{
Profile
(
"TestChromeHeadlessExecutionTime",
10,
() =>
{
var options = new ChromeOptions();
options.AddArguments("headless");
using (IWebDriver driver = new ChromeDriver(options))
{
PerformTestOperations(driver);
}
}
);
}

These are the tests for Chrome, Chrome Headless Mode, Firefox, Edge and PhantomJS.

Benchmark Results

Benchmark Results Headless Browser Chrome

Browser

Average Time (Seconds) 10 executions

Chrome 60 Headless Mode

40.388

Chrome 60

58.509

PhantomJS 2.1

26.282

Firefox 55

85.45

Edge 40.15

11.936

As you can see the results are kind of interesting. The headless mode of Google Chrome performs 30.97% better than the UI version. The other headless driver PhantomJS delivers 34.92% better than the Chrome's headless mode. Surprisingly for me, the Edge browser totally beats all of them. The Firefox is the slowest. 

I have run the tests using Visual Studio 2017 15.3.4 on Windows 10 Enterprise Version 1703 Build 15063.608.

Summary

The Google Chrome headless mode is shipping in Chrome 59. More info.

Firefox Headless Mode? It should be supported by 56+ versions on Windows/Mac (55+ Linux). Currently, it is not working in WebDriver C#, but once it is integrated, I will write an article about it with detailed benchmarks.