Headless Functional Testing with Selenium and PhantomJS

For the purposes of this article, Mocha with Chai have been selected.

Finally, we need a test runner, or an application to run tests by name, and pretty-print the output, while noting how many tests succeeded or failed. This test runner should also offer an assertion library, which allows the coder to express if a test succeeds or fails.

The choice is absolutely free here. There are plenty of JavaScript test runners, but for the purposes of this article, Mocha with Chai have been selected. Mocha provides a considerable amount of flexibility, a wide variety of output formats, and the popular Jasmine-like syntax. Chai allows you to write descriptive BDD-like assertions.

Setup

Here’s the final stack that we’ll be using:

1. Mocha test runner
2. Chai assertion library
3. webdriverjs browser control bindings
4. Selenium browser abstraction and running factory
5. PhantomJS fast headless browser

Node.js and npm

Because most of our stack is based on JavaScript, we need node.js and npm. Both of these are common tools in the community, and I’ll assume that you already have them set up. If you don’t, use the installer on the node.js website. Don’t worry; if anything goes wrong, there are plenty of Node install guides available around the web.

Mocha, Chai and webdriverjs

All three of these can be installed, using npm:

sudo npm install -g mocha chai webdriverjs

Alternatively, you can install them locally in the directory where your tests are located:

npm install mocha chai webdriverjs

Selenium

Download Selenium Server. It is distributed as a single jar file, which you run simply:

java -jar selenium-server-standalone-2.28.0.jar

As soon as you execute this command, it boots up a server to which your testing code will connect later on. Please note that you will need to run Selenium Server every time that you run your tests.

PhantomJS

Quick version

Use npm to install PhantomJS globally:

sudo npm install -g phantomjs

Other options

We require a fresh version of PhantomJS – at least 1.8. This means that packages provided by your package manager (apt-get, MacPorts, …) will most likely be outdated.

You can install using npm without a global installation, or using other methods manually. In this case, however, you will have to tell Selenium where you placed PhantomJS every time you run Selenium:

PATH="/path/to/node_modules/phantomjs/bin:$PATH" java -jar selenium-server-standalone-2.28.0.jar

Combining Everything

Now that we have all the pieces, we have to put everything together.

Remember: before running any tests, you have to run Selenium Server:

java -jar selenium-server-standalone-2.28.0.jar

Selenium will run PhantomJS internally; you don’t have to worry about that.

Now, we need to connect to Selenium from our JavaScript. Here’s a sample snippet, which will initiate a connection to Selenium and have a ready object to control our Selenium instance:

// Use webdriverjs to create a Selenium Clientvar client = require('webdriverjs').remote({    desiredCapabilities: {        // You may choose other browsers        // https://code.google.com/p/selenium/wiki/DesiredCapabilities        browserName: 'phantomjs'    },    // webdriverjs has a lot of output which is generally useless    // However, if anything goes wrong, remove this to see more details    logLevel: 'silent'});client.init();

Now, we can describe our tests and use the client variable to control the browser. A full reference for the webdriverjs API is available in the documentation, but here’s a short example:

client.url('https://example.com/')client.getTitle(function(title){    console.log('Title is', title);});client.setValue('#field', 'value');client.submitForm();client.end();

Let’s use the Mocha and Chai syntax to describe a test; we’ll test some properties of the example.com web page:

describe('Test example.com', function(){    before(function(done) {        client.init().url('https://example.com', done);    });    describe('Check homepage', function(){        it('should see the correct title', function(done) {            client.getTitle(function(title){                expect(title).to.have.string('Example Domain');                done();            });        });        it('should see the body', function(done) {            client.getText('p', function(p){                expect(title).to.have.string(                    'for illustrative examples in documents.'                );                done();            })        });    });    after(function(done) {        client.end();        done();    });});

You might want to share one client initialization over many test files. Create a small Node module to initialize and import it into every test file:

client.js:

exports.client = require('webdriverjs').remote({    // Settings};

test.js:

var client = require('./client').client;var expect = require('chai').expect;// Perform tests

Running

Mocha test suites are execute with the mocha binary. If you followed this guide and installed Mocha locally, then you should describe a full path to the binary yourself: node_modules/mocha/bin/mocha.

By default, Mocha treats any test that takes longer than two seconds as failed. Given that we are actually initializing a web browser and making an HTTP request, we need to increase this timeout to 5 or 10 seconds:

node_modules/mocha/bin/mocha test.js -t 10000

If everything went according to plan, you should see output like this:

  .   1 <span class="nb">test complete</span>

The Next Steps

Once you’ve achieved your desired functional testing results, you may want to consider improving your setup further.

Two obvious directions are continuous integration and distributed Selenium testing.

Continuous integration

Your goal should be to minimize the time that you spend running tests.

You might want to use a full automatic continuous integration server, which will run the tests whenever needed automatically, and inform you if anything goes wrong.

In the world of open source, the role of such a server is covered by Jenkins CI: a convenient, powerful, easy to install service, which will run the tests whenever needed, execute them in any configuration that you provide, and possibly run many more build-related tasks, such as deploying your code to remote servers.

Alternatively, if you feel adventurous, you might experiment with a new project, called GitLab CI, which offers less features, but looks better and is integrated with GitLab, a self hosted GitHub clone.

In any case, your goal should be to minimize the time that you spend running tests. Instead, the tests should be run automatically and should only inform you if anything goes wrong.

Selenium Grid

Selenium has a number of implementation limitations. For example, you cannot run more than a few browsers on the same machine to be tested with Selenium.

In addition, you will notice that, once you have many tests, running all of them can become a lengthy process. Although continuous integration partly alleviates this problem, you might still want to run some tests in parallel on different machines.

Finally, you will soon notice that you want to test different browsers on different operating systems. And, while your testing code can, in theory, talk to different Selenium servers, once you grow a little, this setup needs centralization.

Selenium Grid setup tries to provide exactly that. Instead of having one Selenium server control a bunch of browsers on a machine, you have one Selenium server, which controls multiple Selenium nodes, each which controls only a few browsers on a single operating system.

---

Conclusion

The resulting stack, although not trivial, in reality, is quite simple. The addition of PhantomJS to the Selenium end allows us to begin using Selenium without much initial investment, such as setting up graphical test servers.

The usage of JavaScript as a testing engine ensures that our tests will be kept relevant in the context of web development for the foreseeable future.