Practical End-to-End Testing with Protractor

One of the reasons AngularJS is so great to work with is that it was developed around the idea that testing is so important that it should be built into the framework. Every check-in of the Angular source is tested before it’s accepted into the core.

Testing is incredibly important, especially in a dynamically typed environment like JavaScript. It gives us the opportunity to catch errors before they happen in production. Testing code allows us to be confident about the production value of our code.

As bugs in our code, or any code, are inevitable, it’s important for us to be able to determine where they are and try to eliminate them before they show up in production. Using testing, we can isolate these pieces of functionality in a test environment where we can understand our app from the inside out.

Testing is essential if we are to understand what is happening in our app.

In this article, we’re examining how to test our application using end-to-end testing. End-to-end testing is black box testing. We are testing that the system works as planned from an end user’s perspective.

An end user doesn’t care if a service works as planned; he or she cares that the functionality of our app works as expected. We can think of it as a way to automate starting the app in our browser and clicking through the workflow of the application.

It would be inefficient for us to click through the application manually, so we’ll script our tests to happen automatically.

Protractor

The new, preferred end-to-end testing framework is called Protractor. Unlike the Angular scenario runner, Protractor is built on Selenium’s WebDriver, which is an API, written as extensions, for controlling browsers.

WebDriver has extensions for all sorts of different browsers, including the most popular. We gain speed and stability in our tests by developing against true web browsers.

Luckily, Protractor is built atop the Jasmine framework, so we don’t need to learn a new framework in order to use it. We can also install it as a standalone test runner or embed it in our tests as a library.

Installation

Unlike the Angular scenario runner, Protractor requires a separate standalone server to be running at http://location:4444 (we can configure this location).

Luckily, Protractor itself comes with a tool that eases the installation of a Selenium server.

In order to access the script, we’ll need to install Protractor locally in the top-level directory of the Angular app we want to test.

1
$ npm install protractor

Then we can run the Selenium installation script, located in the local node_modules/ directory:

1
$ ./node_modules/protractor/bin/webdriver-manager update

This script downloads the files required to run Selenium itself and build a start script and a directory with them.

When this script is finished, we can start the standalone version of Selenium with the Chrome driver by executing the start script:

1
$ ./node_modules/protractor/bin/webdriver-manager start

If you are having trouble running your Selenium installation, try updating the ChromeDriver by downloading the latest version here.

Now we can use Protractor to connect to our Selenium server, which is running in the background.

Configuration

Like Karma, Protractor itself requires a configuration script that tells Protractor how to run, how to connect to Selenium, etc.

The easiest method for creating a configuration file for Protractor is to copy a reference configuration file from the installation directory.

1
$ cp ./node_modules/protractor/example/chromeOnlyConf.js protractor_conf.js

In order to get Protractor running, we need to make a few modifications to the script. First, the default configuration script uses a Chrome driver that doesn’t exist in our current directory. Instead, we need to point it to the ChromeDriver in the local ./node_modules.

1
chromeDriver: './node_modules/protractor/selenium/chromedriver',

Next, we need to point the specs array to our local tests.

1
specs: ['test/e2e/**/*_spec.js'],

There are quite a few options for setting up our tests using Protractor. Although we’ll cover a few here, keep in mind that we have a lot of options in configuring Protractor.

We have two options for running Protractor tests, the first of which is to use Protractor to start Selenium when we run our Protractor tests. This option is called standalone mode. The example Protractor configuration file that we copied includes this setup.

1
2
chromeOnly: true, chromeDriver: './node_modules/protractor/selenium/chromedriver',

The second method for running Protractor tests is to connect to a separately running Selenium server. When our tests start to grow more complex, we’ll likely want to run our tests using a separate Selenium server.

To configure Protractor to use this separate server, we need to delete the previous two options (chromeOnly and chromeDriver) and add the seleniumAddress option that points to the running Selenium server.

1
seleniumAddress: 'http://0.0.0.0:4444/wd/hub',

Testing

When we start testing with Protractor, we set up our tests to work through Jasmine. That is, we simply start writing our tests like we do when we write our Karma tests. For instance, a simple Protractor test setup might look like:

1
2
3
4
5
6
7
8
9
10
describe('homepage', function() { beforeEach(function() { // before function }); it('should load the page', function() { // test goes here expect(...).toEqual('hello'); }); });

Although the content of the tests above isn’t complete, the structure is familiar: It’s set up using Jasmine. We get to use the beforeEach() and afterEach() functions as well as nested describe() blocks for the structure around our tests.

To actually implement tests, we’ll use the same expect() syntax that Jasmine gives us.

Writing tests for Protractor requires us to work with a few global variables that Protractor exposes to us in our tests. The following is a list of a few of those global variables:

browser

The browser variable is a wrapper around the WebDriver instance. We use the browser variable for any navigation or for grabbing any information off the page.

We can use the browser variable to navigate to a page using the get() function:

1
2
3
beforeEach(function() { browser.get('http://127.0.0.1:9000/'); });

We can run some neat tricks with the browser object. For instance, we can debug the page using the debugger() method on the browser object:

1
2
3
4
5
6
7
it('should find title element', function() { browser.get('app/index.html'); browser.debugger(); element(by.binding('user.name')); });

To kick this test off with the node debugger, we can run the test in debug mode:

1
$ protractor debug conf.js

We get benefits when we run Protractor in debug mode: The execution halts in the browser, and every client-side script provided by Protractor is available in the console.

To get access to the Protractor client-side scripts, we can call them with the window.clientSideScripts object that’s inserted by Protractor.

Let’s Test!

Although it’s easy to talk about how to use Protractor, it’s not always clear how to get it going. As we always try to share the highest-quality material available on Angular, we’re going to dive right into testing an application and strategies.

Our Application

Let’s suppose we have an application that provides an alternative view for viewing GitHub issues. The simple app itself has only a few major features:

The final app we’re building looks like this:

With any application, we can strategize about the tests that we want to write. We may end up writing a hundred tests for an incredibly simple app OR we might end up writing very few. Finding the right balance between the two will give us an advantage when it comes time to implement both the application and the tests.

Strategy for Testing

We’ve found that the best balance between writing tests and writing code pretty much exists in knowing what to test just as much as how to test. Whenever we’re writing tests for our code, we want to be writing tests that specifically address the behavior we are implementing. That is, we don’t need to write a test to make sure an <h1> tag’s content changes as we type into an <input> field. We do need to test our custom filtering for live search, for instance.

We’ve found that writing our tests ahead of time when prototyping never works out in our favor. When we’re in the prototyping phase, we’ll write very few tests, if any at all, as we’re still working through features of our application. When the application starts to grow, however, writing tests is always a good idea to ensure that the app behaves as we expect it to behave in production.

Finally, we’ll want to set up our tests such that each block tests as little as possible. Ideally, each test block should contain no more than 1 expectation.

Enough theory, let’s apply some strategy to testing our app.

We’ll want to test that our page updates with the title of the new repo against which we’re testing. The Angular app employs a custom service that makes an $http request to github.com. This request comes back, and we fill the rest of the front page.

Second, we’ll want to test that the page navigation and content change. This test will involve us pressing the navigation buttons in our view to prompt a $location change.

Let’s get started!

Setting Up Our First Tests

Our Protractor configuration file is pretty simple and nearly unmodified from the example configuration that comes with Protractor itself:

1
2
3
4
5
6
7
8
9
10
// An example configuration file. exports.config = { seleniumAddress: 'http://0.0.0.0:4444/wd/hub', capabilities: { 'browserName': 'chrome' }, specs: ['test/e2e/**/*.spec.js'], jasmineNodeOpts: { showColors: true, defaultTimeoutInterval: 30000 } };

We’ll be writing our Protractor tests in the test/e2e directory, as we specified in our configuration file with the naming convention of [name].spec.js. Let’s create our first test in the test/e2e directory called main.spec.js.

As Protractor tests are simply Jasmine tests, we’ll start out with a simple Jasmine stub:

1
2
3
4
// in test/e2e/main.spec.js describe('E2E: main page', function() { // Our tests go in here });

Given that we’re writing our tests with Jasmine, we can use our beforeEach() block to set them up. We also need to keep track of the Protractor instance, so let’s set up a variable we’ll call ptor that will hold on to it. For every single one of these tests, we’ll use the browser object to navigate to the home page.

As with end-to-end tests, we need to have a server running against which our end-to-end tests run.

1
2
3
4
5
6
7
8
9
describe('E2E: main page', function() { var ptor; beforeEach(function() { browser.get('http://127.0.0.1:9000/'); ptor = protractor.getInstance(); }); });

Instead of pointing our tests to the full URL every time we want to test a page, we can set the baseUrl in our Protractor configuration file. For the rest of the section, we’ll assume that we have the option set in our config file like so:

1
baseUrl: 'http://127.0.0.1:9000/',

Our first test will simply test that the main page loads up: We can test that an element exists on the page. Since our home page contains the ID #home, we can write an expectation to guarantee the condition.

We’ll first find the element we’re interested in using the by.id() function to target the <div> with the #main ID:

1
2
3
it('should load the home page', function() { var ele = by.id('home'); });

Once we have the element, we can set an expectation that the element is present on the page using the Protractor instance’s method isElementPresent():

1
2
3
4
it('should load the home page', function() { var ele = by.id('home'); expect(ptor.isElementPresent(ele)).toBe(true); });

To run our tests, we need to launch a Selenium server. Luckily, Protractor makes this process easy using a built-in tool called webdriver-manager. This manager is included by default (as we saw above). Let’s start our webdriver-manager:

1
$ ./node_modules/protractor/bin/webdriver-manager start

In a new shell, we’ll need to launch Protractor to actually run our tests. The protractor binary takes a single argument: the configuration file:

1
$ ./node_modules/protractor/bin/protractor protractor_conf.js

Testing the Input Box

First, we’re setting our sights on testing the input box. The main page loads a single form with a single input box that is only shown if the user has not yet picked a repo in which to search issues. The <input type="text"> is bound to a model called repoName. Once the user has submitted the form, then the form itself disappears and the list of issues appears instead.

The HTML looks like:

1
2
3
4
5
6
7
8
9
10
<div id="repoform" class="main" ng-if="!repoName"> <form ng-submit="getIssues()" class="input-group"> <div class="input-group"> <input type="text" ng-model='repo.name' placeholder='Enter repo name' /> <span class="input-group-btn"> <input type="submit" class="btn btn-primary" value="Search"> </span> </div> </form> </div>

The functionality that we’re interested in testing is that the form disappears and the listing appears. In a new test, we’ll want to target the <input> element and write into it. We can do so with the sendKeys() method on our targeted element. To target our <input> element, we can use the by.input() method that gives us access to find input elements containing a binding with ng-model.

1
2
3
it('the input box should go away on submit', function() { element(by.input('repo.name')).sendKeys('angular/angular.js\n'); });

When we run this test, we’ll see that the <input> field is filled out. There will be no expectations set up as we haven’t yet written one, but we can see the input filling up with angular/angular.js.

To make our input field disappear, we need to submit the form. The easiest way to submit a form is by faking pressing the enter button. In the above sendKeys(), we included the \n character, which fakes pressing enter in the <input> element.

At this point, we only need to set up an expectation that the repoform element no longer exists on the page (as we’re hiding it with ng-if). We can use the same method we did above to confirm it is no longer on the page:

1
2
3
4
it('the input box should go away on submit', function() { element(by.input('repo.name')).sendKeys('angular/angular.js\n'); expect(ptor.isElementPresent(by.id('repoform'))).toBe(false); });

Testing the Listing

After we’ve set up our tests that set up the <input> element, we can move on and test the functionality of our listing page.

Since the rest of the tests that we’ll write will take place in the listing page, we’ll nest the rest of the tests here within their own describe() block. Using a separate block allows us to set another beforeEach() block where we’ll set up the tests to run against the angular/angular.js github repository.

The describe block will simply act as a user coming to our homepage and typing in the input field and pressing enter. It might seem superfluous to set up our tests in the manner, but remember that the goal of end-to-end testing is to automate user interaction.

1
2
3
4
5
6
7
describe('listing page', function() { beforeEach(function() { element(by.input('repo.name')).sendKeys('angular/angular.js\n'); }); // ... // our listing page tests will go here });

The listing page will have a number of elements that we’ll iterate over using ng-repeat. Using the GitHub APi, we’ll retrieve 30 issues by default. We can test that this page does in fact resolve to 30 issues.

We can target the ng-repeat element by using the by.repeater() helper. This helper looks at the ng-repeat directives on the page and finds the one that matches our expression. In this case, we’re repeating using the Angular expression d in data | orderBy:created_at:false. We can target the repeater using:

1
by.repeater('d in data | orderBy:created_at:false')

We have the option of being explicit with the filters (as we have done above) or leaving off the filters and being more generic:

1
by.repeater('d in data');

Using the by.repeater() doesn’t actually return us any elements; only a pointer to the method to fetch elements. If we try to set expectations against the object returned by the by.repeater() method, we’ll simply get an ugly error. Protractor is set up in this way because elements are populated through promises, so we have to use the element.all() function to give us access to the resolved elements.

1
var elems = element.all(by.repeater('d in data'));

With our element targeted, we can simply ask for a count on the element.all() object and set an expectation that there will be 30 elements:

1
2
3
4
it('should have 30 issues', function() { var elems = element.all(by.repeater('d in data')); expect(elems.count()).toBe(30); });

Great. Let’s dive more deeply into these repeated elements and ensure that the avatar is shown for each of them. We can make a reasonable assumption that the each element is a repeat of every other element, so we’ll set up a test that tests a single element.

In order to fetch the elements on the page, we’ll start in the same manner using the by.repeater() method. The element.all() method returns an object containing several methods we can use to interact with the repeated listing elements. In this case, we’ll simply use the first() method to find the first element in the list.

Since the list is not yet populated on the page, the first() method returns a promise that will be resolved with the first element on the page.

1
2
3
4
5
6
it('includes a user gravatar per-element', function() { var elems = element.all(by.repeater('d in data')); elems.first().then(function(elm) { // elm is the first element }); });

Since we’re interested in only a single child element, we’ll use the findElement() method to fetch the <img> element. We can target this element by multiple methods. We’ll use the by.tagName() method. As with the first() method, findElement() returns a promise for the very same reason.

1
2
3
4
5
6
7
8
it('includes a user gravatar per-element', function() { var elems = element.all(by.repeater('d in data')); elems.first().then(function(elm) { elm.findElement(by.tagName('img')).then(function(img) { // img is the <img> element }); }); });

We’re particularly interested in making sure the src attribute includes a Gravatar URL. We can dive even more deeply into the details of the element using various methods provided by the element object. In this case, we’ll use the getAttribute() method to find the src attribute. As with the previous two methods, we’ll need to set it up as a promise:

1
2
3
4
5
6
7
8
9
10
it('includes a user gravatar per-element', function() { var elems = element.all(by.repeater('d in data')); elems.first().then(function(elm) { elm.findElement(by.tagName('img')).then(function(img) { img.getAttribute('src').then(function(src) { // src is the text source }); }); }); });

Now that we have the src attribute, we can set up an expectation that the source matches gravatar.com, as GitHub uses Gravatar:

1
2
3
4
5
6
7
8
9
10
it('includes a user gravatar per-element', function() { var elems = element.all(by.repeater('d in data')); elems.first().then(function(elm) { elm.findElement(by.tagName('img')).then(function(img) { img.getAttribute('src').then(function(src) { expect(src).toMatch(/gravatar\.com\/avatar/); }); }) }); });

Testing Routing

The last piece of functionality we’ll want to test is the page navigation. As expected, we’ll set up our tests using actions on elements of the page. In this case, we’ll target the /about link using CSS and click on the link.

The HTML looks like:

1
2
3
4
5
6
7
<div class="header"> <ul class="nav"> <li ng-class="{'active': isCurrentPage('')}"><a id="homelink" ng-href="#">Home</a></li> <li ng-class="{'active': isCurrentPage('about')}"><a id='aboutlink' ng-href="#/about">About</a></li> </ul> <h3 class="text-muted">protractorer</h3> </div>

The /about link is the second element in the header.nav list. The quickest method to target the list is to use a CSS selector through the by.css() method.

1
2
3
it('should navigate to the /about page when clicking', function() { var link = element(by.css('.header ul li:nth-child(2)')) });

Now that we have the link, we can click on it to navigate to the new URL. Once we’ve navigated to the /about page, we can test that the page content shows up, or we can test that the current route contains the /about path. Since we can reasonably expect the Angular router to work, we can assume the about page content will load if the browser’s URL matches the about page.

Therefore, we’ll simply test that the current URL includes /about. We can get hold of the current URL using the Protractor instance method getCurrentUrl():

1
2
3
4
it('should navigate to the /about page when clicking', function() { element(by.css('.header ul li:nth-child(2)')).click(); expect(ptor.getCurrentUrl()).toMatch(/\/about/); });

Finally, since we’re testing the front end, we can also expect that the active class will be added to the link.

The active class adds the color style on the button.

We’ll want to run the same action we did previously where we click on the /about link. Any time that we find ourselves duplicating test data, it’s usually a good idea to nest the tests in their own describe block and move the duplication into the block. Let’s go ahead and move our tests into the describe block:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
describe('page navigation', function() { var link; beforeEach(function() { link = element(by.css('.header ul li:nth-child(2)')); link.click(); }); it('should navigate to the /about page when clicking', function() { expect(ptor.getCurrentUrl()).toMatch(/\/about/); }); it('should add the active class when at /about', function() { // should have the active class }); });

The final test tests whether the class list contains the string active:

1
expect(link.getAttribute('class')).toMatch(/active/);

Want More?

Protractor is a highly active project on GitHub and an incredibly powerful end-to-end testing framework. It will soon replace Karma as the official end-to-end test runner and become the official testing framework for Angular.

The testing source code is available at http://j.mp/1m4xdma.

If you enjoyed this section, check out ng-book.com for more details.

Thanks, and happy testing!

Get the weekly email all focused on AngularJS. Sign up below to receive the weekly email and exclusive content.
We will never send you spam and it's a cinch to unsubscribe.

Download a free sample of the ng-book: The Complete Book on AngularJS

ng-book: The Complete Book on AngularJS is the canonical AngularJS book available today.

It's free, so just enter your email address and the PDF will be sent directly to your inbox. Mailchimp can take up to an hour to deliver the free sample chapter, but if you don't receive it within the hour, send us an email and we'll manually send them to you!

We'll send you updates about the book, when it updates and other free content.

We will never send you spam and it's a cinch to unsubscribe.

Comments

comments powered by Disqus