NightWatch

What is Nightwatch?

Nightwatch.js is an automated testing framework for web applications and websites, written in Node.js and using the W3C WebDriver API (formerly Selenium WebDriver).

It is a complete browser (End-to-End) testing solution which aims to simplify the process of setting up Continuous Integration and writing automated tests. Nightwatch can also be used for writing Node.js unit tests.

Nightwatch got its name from the famous painting The Night Watch by Dutch artist Rembrandt van Rijn. The masterpiece is prominently displayed in the Rijksmuseum, in Amsterdam – The Netherlands.

Overview of WebDriver

WebDriver is a general purpose library for automating web browsers. It was started as part of the Selenium project, which is a very popular and comprehensive set of tools for browser automation, initially written for Java but now with support for most programming languages.

Nightwatch uses the WebDriver API to perform the browser automation related tasks, like opening windows and clicking links for instance.

WebDriver is now a W3C specification, which aims to standardize browser automation. WebDriver is a remote control interface that enables introspection and control of user agents. It provides a platform and a restful HTTP api as a way for web browsers to be remotely controlled.

Theory of Operation

Nightwatch works by communicating over a restful HTTP api with a WebDriver server (typically the Selenium server). The restful API protocol is defined by the W3C WebDriver API. See below for an example workflow for browser initialization.

Most of the times, Nightwatch needs to send at least 2 requests to the WebDriver server in order to perform a command or assertion, the first one being the request to locate an element given a CSS selector (or Xpath expression) and the next to perform the actual command/assertion on the given element.

Install Node.js

From nodejs.org:

“Node.js is a platform built on Chrome’s JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.”

There are installation packages and instructions for most major Operating systems on its website nodejs.org. Remember to install also the npm tool, which is the node package manager and is distributed with the Node.js installer.

Install Nightwatch

To install the latest version using the npm command line tool, run the following:

$ npm install [-g] nightwatch

Add -g option to make nightwatch runner available globally in your system.

Selenium Server Setup

The most common WebDriver implementation is the Selenium Server. This allows you to manage multiple browser configurations in one place. However, you can also run the individual browser drivers directly, such as the ChromeDriver, more details are available in the Browser Drivers Setup section.

Selenium Server

Selenium Server is a Java application which Nightwatch uses to connect to the various browsers. It runs separately on the machine with the browser you want to test. You will need to have the Java Development Kit (JDK) installed, minimum required version is 7. You can check this by running java -version from the command line.

Download Selenium

Download the latest version of the selenium-server-standalone-{VERSION}.jar file from the Selenium downloads page and place it on the computer with the browser you want to test. In most cases this will be on your local machine and typically inside your project’s source folder.

A good practice is to create a separate subfolder (e.g. bin) and place it there as you might have to download other driver binaries if you want to test multiple browsers.

Running Selenium Automatically

If the server is on the same machine where Nightwatch is running, it can be started/stopped directly by the Nightwatch Test Runner.

Running Selenium Manually

To run the Selenium Server manually, from the directory with the jar run the following:

$ java -jar selenium-server-standalone-{VERSION}.jar

Using Selenium

For viewing all the run-time options, run the previous command adding the -help:

$ java -jar selenium-server-standalone-{VERSION}.jar -help

Starting with Selenium 3, FirefoxDriver is no longer included in the package. Also, starting with version 48, Firefox is no longer compatible with FirefoxDriver which is shipped with Selenium 2.x. Firefox users are advised to use GeckoDriver for their testing. For more info, refer to the browser setup section.

More info about running the Selenium Server can be found here: https://github.com/SeleniumHQ/selenium/wiki/RemoteWebDriverServer

The test runner expects a configuration file to be passed, using by default a nightwatch.json file from the current directory, if present. A nightwatch.conf.js file will also be loaded by default, if found.

Let’s create the nightwatch.json in the project’s root folder and add this inside:

{
  "src_folders" : ["tests"],
  "output_folder" : "reports",
  "custom_commands_path" : "",
  "custom_assertions_path" : "",
  "page_objects_path" : "",
  "globals_path" : "",

  "selenium" : {
    "start_process" : false,
    "server_path" : "",
    "log_path" : "",
    "port" : 4444,
    "cli_args" : {
      "webdriver.chrome.driver" : "",
      "webdriver.gecko.driver" : "",
      "webdriver.edge.driver" : ""
    }
  },

  "test_settings" : {
    "default" : {
      "launch_url" : "http://localhost",
      "selenium_port"  : 4444,
      "selenium_host"  : "localhost",
      "silent": true,
      "screenshots" : {
        "enabled" : false,
        "path" : ""
      },
      "desiredCapabilities": {
        "browserName": "firefox",
        "marionette": true
      }
    },

    "chrome" : {
      "desiredCapabilities": {
        "browserName": "chrome"
      }
    },

    "edge" : {
      "desiredCapabilities": {
        "browserName": "MicrosoftEdge"
      }
    }
  }
}

Using both configuration files is also possible, with nightwatch.conf.js always taking precedence if both are found.

Example

module.exports = (function(settings) {
  settings.test_workers = false;
  return settings;
})(require('./nightwatch.json'));

Basic settings

Selenium settings

Below are a number of options for the selenium server process. Nightwatch can start and stop the Selenium process automatically which is very convenient as you don’t have to manage this yourself and focus only on the tests.

If you’d like to enable this, set start_process to true and specify the location of the jar file inside server_path.

Test settings

Below are a number of settings that will be passed to the Nightwatch instance. You can define multiple sections (environments) of test settings so you could overwrite specific values per environment.

A “default” environment is required. All the other environments are inheriting from default and can overwrite settings as needed.

{
  ...
  "test_settings" : {
    "default" : {
      "launch_url" : "http://localhost",
      "globals" : {
        "myGlobalVar" : "some value",
        "otherGlobal" : "some other value"
      }
    },

    "integration" : {
      "launch_url" : "http://staging.host",
      "globals" : {
        "myGlobalVar" : "other value"
      }
    }
  }
}

The key of the settings group can be passed then to the runner as the --env argument to use the specified settings, like so:

$ nightwatch --env integration

This can be useful if you need to have different settings for your local machine and the Continuous Integration server.

The launch_url property

This property will be made available to the main Nightwatch api which is used in the tests. Its value depends on which environment is used.

If you run your tests as in the example above (with --env integration) launch_url will be set to http://staging.host, as per the configuration. Otherwise it will have the value defined in the defaultenvironment (i.e. http://localhost).

module.exports = {
  'Demo test' : function (browser) {
    browser
      .url(browser.launchUrl)
      // ...
      .end();
  }
};

Test globals

A very useful concept that Nightwatch provides is test globals. In its most simple form, this is a dictionary of name-value pairs which is defined in your nightwatch.json configuration file. Like the launch_url property, this is made available directly on the Nightwatch api which is passed to the tests. It is also dependent on the environment used, having the ability to overwrite specific globals per environment.

If we still pass the --env integration option to the runner, then our globals object will look like below:

module.exports = {
  'Demo test' : function (browser) {

    console.log(browser.globals);
    // {
    //   "myGlobalVar" : "some value",
    //   "otherGlobal" : "some other value"
    // }

  }
};

By default, a deep object copy will be created for each test suite run. If you’d like to maintain the same object throughout the entire tests run, set the persist_globals option to true, as detailed below.

Full list of settings

This section contains guides for getting started with most of the major browsers and setup instructions on how to configure the individual webdriver implementations to work with Nightwatch.

The individual drivers described here are usually standalone applications which are used to interact with the browsers via the WebDriver HTTP API. You can run them either directly, or through the Selenium Server.

GeckoDriver

Overview

GeckoDriver is a standalone application used to interact with Gecko-based browsers, such as Firefox. It is written in Rust and maintained by Mozilla.

Starting with Firefox 48, GeckoDriver is the only way to automate Firefox, the legacy FirefoxDriver which used to be part of Selenium is no longer supported. Internally it translates the HTTP calls into Marionette, Mozilla’s automation protocol built into Firefox.

Download

Binaries are available for download on the GeckoDriver Releases page on GitHub, for various platforms.

Selenium 2.x users are advised to use version v0.9, whereas Selenium 3 users should use the latest version.

Usage

If you’re using GeckoDriver through Selenium Server, simply set the cli argument "webdriver.gecko.driver" to point to the location of the binary file. E.g.:

{
  "selenium" : {
    "start_process" : true,
    "server_path" : "./bin/selenium-server-standalone-3.{VERSION}.jar",
    "log_path" : "",
    "port" : 4444,
    "cli_args" : {
      "webdriver.gecko.driver" : "./bin/geckodriver"
    }
  }
}

GeckoDriver can also be used as a standalone application. Usage steps are documented on GitHub: https://github.com/mozilla/geckodriver#usage.

Command line usage

$ ./bin/geckodriver-0.10 -help
geckodriver 0.10.0

USAGE:
    geckodriver-0.10 [FLAGS] [OPTIONS]

FLAGS:
        --connect-existing    Connect to an existing Firefox instance
    -h, --help                Prints help information
        --no-e10s             Start Firefox without multiprocess support (e10s) enabled
    -V, --version             Prints version information
    -v                        Set the level of verbosity. Pass once for debug level logging and twice for trace level logging

OPTIONS:
    -b, --binary            Path to the Firefox binary, if no binary capability provided
        --log                Set Gecko log level [values: fatal, error, warn, info, config, debug, trace]
        --marionette-port     Port to use to connect to gecko (default: random free port)
        --host                Host ip to use for WebDriver server (default: 127.0.0.1)
    -p, --port                Port to use for WebDriver server (default: 4444)

Firefox Capabilities

GeckoDriver supports a capability named firefoxOptions which takes Firefox-specific preference values. Details are available on the GeckoDriver GitHub page: https://github.com/mozilla/geckodriver#firefox-capabilities.

Firefox Profile

Specifying the firefox profile can be done by setting the profile property in the firefoxOptions dictionary, as detailed above. This can be the base64-encoded zip of a profile directory and it may be used to install extensions or custom certificates.

Implementation Status

GeckoDriver is not yet feature complete, which means it does not yet offer full conformance with the WebDriver standard or complete compatibility with Selenium. Implementation status can be tracked on the Marionette MDN page.

ChromeDriver

Overview

ChromeDriver is a standalone server which implements the W3C WebDriver wire protocol for Chromium. ChromeDriver is available for Chrome on Android and Chrome on Desktop (Mac, Linux, Windows and ChromeOS).

Download

Binaries are available for download on the ChromeDriver Downloads page, for various platforms.

Selenium Server Usage

If you’re using ChromeDriver through Selenium Server, simply set the cli argument "webdriver.chrome.driver" to point to the location of the binary file. E.g.:

{
  "selenium" : {
    "start_process" : true,
    "server_path" : "./bin/selenium-server-standalone-3.{VERSION}.jar",
    "log_path" : "",
    "port" : 4444,
    "cli_args" : {
      "webdriver.chrome.driver" : "./bin/chromedriver"
    }
  }
}

Standalone Usage

If you’re only running your tests against Chrome, running ChromeDriver standalone is easier and slightly faster. Also there is no dependency on Java.

This requires a bit more configuration:

1) First, disable Selenium Server, if applicable:

{
  "selenium" : {
    "start_process" : false
  }
}

2) Configure the port and default path prefix.

ChromeDriver runs by default on port 9515. We also need to clear the default_path_prefix, as it is set by default to /wd/hub, which is what selenium is using.

{
  "test_settings" : {
    "default" : {
      "selenium_port"  : 9515,
      "selenium_host"  : "localhost",
      "default_path_prefix" : "",

      "desiredCapabilities": {
        "browserName": "chrome",
        "chromeOptions" : {
          "args" : ["--no-sandbox"]
        },
        "acceptSslCerts": true
      }
    }
  }
}

3) Start the ChromeDriver server

The easiest way to manage the ChromeDriver process is by using the chromedriver NPM package, which is a third-party wrapper against the binary. This will abstract the downloading of the chromedriver binary and will make it easy to manage starting and stopping of the process.

You can add this to your external globals file, like so:

var chromedriver = require('chromedriver');
module.exports = {
  before : function(done) {
    chromedriver.start();

    done();
  },

  after : function(done) {
    chromedriver.stop();

    done();
  }
};

Using a fixed ChromeDriver version

In some situations you may need to use a specific version of ChromeDriver. For instance, the CI server runs an older version of Chrome. Then you will need an older version of ChromeDriver.

Here’s what your globals file might look like:

var chromedriver = require('chromedriver');
var path = require('path');
var driverInstanceCI;

function isRunningInCI() {
  return this.test_settings.globals.integration;
}

function startChromeDriver() {
  if (isRunningInCI.call(this)) {
    var location = path.join(__dirname, '../bin/chromedriver-linux64-2.17');
    driverInstanceCI = require('child_process').execFile(location, []);
    return;
  }

  chromedriver.start();
}

function stopChromeDriver() {
  if (isRunningInCI.call(this)) {
    driverInstanceCI && driverInstanceCI.kill();
    return;
  }

  chromedriver.stop();
}

module.exports = {
  'ci-server' : {
    integration : true
  },

  before : function(done) {
    startChromeDriver.call(this);

    done();
  },

  after : function(done) {
    stopChromeDriver.call(this);

    done();
  }
};

Run your tests then with (on the CI server):

$ ./node_modules/.bin/nightwatch --env ci-server

ChromeOptions

You can specify Chrome options or switches using the chromeOptions dictionary, under the desiredCapabilities. Refer to the ChromeDriver website for a fill list of supported capabilities and options.

Command line usage

$ ./bin/chromedriver -h
Usage: ./bin/chromedriver [OPTIONS]

Options
  --port=PORT                     port to listen on
  --adb-port=PORT                 adb server port
  --log-path=FILE                 write server log to file instead of stderr, increases log level to INFO
  --verbose                       log verbosely
  --version                       print the version number and exit
  --silent                        log nothing
  --url-base                      base URL path prefix for commands, e.g. wd/url
  --port-server                   address of server to contact for reserving a port
  --whitelisted-ips               comma-separated whitelist of remote IPv4 addresses which are allowed to connect to ChromeDriver

Microsoft WebDriver

Overview

Microsoft WebDriver is a standalone server which implements the W3C WebDriver wire protocol for the Edge browser. It is supported by Windows 10 and onwards.

Download

Binaries are available for download on the Microsoft WebDriver homepage.

Selenium Server Usage

If you’re using Microsoft WebDriver through Selenium Server, simply set the cli argument "webdriver.edge.driver"to point to the location of the binary file. E.g.:

{
  "selenium" : {
    "start_process" : true,
    "server_path" : "bin/selenium-server-standalone-3.{VERSION}.jar",
    "log_path" : "",
    "port" : 4444,
    "cli_args" : {
      "webdriver.edge.driver" : "bin/MicrosoftWebDriver.exe"
    }
  },
  "test_settings" : {
    "default" : {
      "selenium_port"  : 4444,
      "selenium_host"  : "localhost",

      "desiredCapabilities": {
        "browserName": "MicrosoftEdge",
        "acceptSslCerts": true
      }
    }
  }
}

Standalone Usage

If you’re only running your tests against Edge, running the EdgeDriver standalone can be slightly faster. Also there is no dependency on Java.

This requires a bit more configuration and you will need to start/stop the EdgeDriver:

1) First, disable Selenium Server, if applicable:

{
  "selenium" : {
    "start_process" : false
  }
}

2) Configure the port and default path prefix.

Microsoft WebDriver runs by default on port 9515. We also need to clear the default_path_prefix, as it is set by default to /wd/hub, which is what selenium is using.

{
  "test_settings" : {
    "default" : {
      "selenium_port"  : 17556,
      "selenium_host"  : "localhost",
      "default_path_prefix" : "",

      "desiredCapabilities": {
        "browserName": "MicrosoftEdge",
        "acceptSslCerts": true
      }
    }
  }
}

3) Start the MicrosoftWebDriver server

From the Windows CMD prompt, simply CD to the folder where the MicrosoftWebDriver.exe binary is located and run:

C:\nightwatch\bin>MicrosoftWebDriver.exe
[13:44:49.515] - Listening on http://localhost:17556/

Full command line usage:

C:\nightwatch\bin>MicrosoftWebDriver.exe -h
Usage:
 MicrosoftWebDriver.exe --host= --port= --package= --verbose

Implementation Status

EdgeDriver is not yet feature complete, which means it does not yet offer full conformance with the WebDriver standard or complete compatibility with Selenium. Implementation status can be tracked on the Microsoft WebDriver homepage.