Tuesday, March 31, 2015

Automated Polymer Testing with Grunt and web-component-tester


Today, I try my hand at combining Grunt and web-component-tester for local Polymer development.

Last night's experiment with Gulp was reasonable successful. Initial setup with Gulp and web-component-tester (WCT) was brilliantly easy. Even the first bits of customization was relatively simple. Quickly, however, I reached a point where I was hip-deep in node.js streams, error handling and the alphabet soup of NPM packages that Gulp uses to make sense of them.

Grunt will certainly require a little more initial effort, but I am all-but-certain that I can get my complete local setup in place. The best laid plans....

What I want is a means to automatically run my tests locally whenever I make a change to code or test files. I also a appreciate a little LiveReload support for my sample pages. I have done both with Grunt before, though not with web-component-tester.

I start by wiping my sample application's package.json file so that I can re-add Grunt, web-component-tester, and a couple of other dependencies that I know that I will want to use:
$ npm install grunt grunt-notify  grunt-contrib-watch web-component-tester --save-dev
This leaves me with a package.json that contains:
{
  "name": "wct_sample",
  "devDependencies": {
    "grunt": "^0.4.5",
    "grunt-contrib-watch": "^0.6.1",
    "grunt-notify": "^0.4.1",
    "web-component-tester": "^2.2.6"
  }
}
I start my Gruntfile.js configuration file with the sample included in the WCT homepage:
module.exports = function(grunt) {
  pkg: grunt.file.readJSON('package.json'),
  grunt.initConfig({
    'wct-test': {
      local: {
        options: {remote: false}
      },
      remote: {
        options: {remote: true}
      },
      chrome: {
        options: {browsers: ['chrome']}
      }
    }
  });

  grunt.loadNpmTasks('web-component-tester');
};
That gives me wct-test:local, wct-test:remote, and wct-test:chrome tasks. The first is the one that will run my test suite (currently just a single test):
$ grunt wct-test:local:chrome
Running "wct-test:local:chrome" (wct-test) task
Starting Selenium server for local browsers
Selenium server running on port 33822
Web server running on port 35707 and serving from /home/chris/repos/polymer-book/play/web-component-tester
chrome 41                Beginning tests via http://localhost:35707/js/test/index.html?cli_browser_id=0
chrome 41                Tests passed
firefox 36               Beginning tests via http://localhost:35707/js/test/index.html?cli_browser_id=1
firefox 36               Tests passed
Test run ended with great success

chrome 41 (1/0/0)                       firefox 36 (1/0/0)                    

Done, without errors.
Nice. That was just as easy to get running as the Gulp version (admittedly there is less configuration with Gulp for the very simple case).

Next, I would like to run my test suite whenever there is a change to any file in the tests or elements directories. This is why I included the grunt-contrib-watch development dependency at the outset. I update Gruntfile.js to include the watch tasks and definitions:
module.exports = function(grunt) {
  pkg: grunt.file.readJSON('package.json'),
  grunt.initConfig({
    'wct-test': {
      local: {
        options: {remote: false}
      },
      // ...
    },
    watch: {
      code: {
        files: ['index.html', 'elements/**'],
        options: {
          livereload: true
        }
      },
      tests: {
        files: ['elements/**', 'test/**'],
        tasks: ['wct-test:local'],
        options: {atBegin: true}
      }
    }
  });

  grunt.loadNpmTasks('web-component-tester');
  grunt.loadNpmTasks('grunt-contrib-watch');
};
The watch:code task will watch the specified files which impact the sample page. On update, they trigger the browser to LiveReload the page (if the extension is being used). The watch:tests task will watch code and test files for changes, running the WCT's wct-test:local task when it sees those changes. For good measure, I specify that the tests should be run at start as well.

And that all works nicely. When I invoke watch, my tests are run right away:
$ grunt watch
Running "watch" task
Waiting...
Running "wct-test:local" (wct-test) task
Starting Selenium server for local browsers
Selenium server running on port 44706
Web server running on port 37026 and serving from /home/chris/repos/polymer-book/play/web-component-tester
chrome 41                Beginning tests via http://localhost:37026/js/test/index.html?cli_browser_id=0
chrome 41                Tests passed
firefox 36               Beginning tests via http://localhost:37026/js/test/index.html?cli_browser_id=1
firefox 36               Tests passed
Test run ended with great success

chrome 41 (1/0/0)                       firefox 36 (1/0/0)                    

Done, without errors.
Completed in 7.828s at Tue Mar 31 2015 23:37:56 GMT-0400 (EDT) - Waiting...
This task does not end, but polls for changes to the specified files. If I make a change to one of them, I find:
>> File "test/index.html" changed.
Running "wct-test:local" (wct-test) task
Starting Selenium server for local browsers
Selenium server running on port 49367
Web server running on port 41348 and serving from /home/chris/repos/polymer-book/play/web-component-tester
chrome 41                Beginning tests via http://localhost:41348/js/test/index.html?cli_browser_id=0
chrome 41                Tests passed
firefox 36               Beginning tests via http://localhost:41348/js/test/index.html?cli_browser_id=1
firefox 36               Tests passed
Test run ended with great success

chrome 41 (1/0/0)                       firefox 36 (1/0/0)                    

Done, without errors.
Completed in 7.491s at Tue Mar 31 2015 23:38:18 GMT-0400 (EDT) - Waiting...
To complete my desired setup, I would like to have desktop notifications fire at the end of each test run. This is where the grunt-notify package comes in handy. I want to use "notify hooks” in this case. Whenever a task completes successfully, these hooks will send a desktop notification with the result message.

To use hooks, I have to load the grunt-notify tasks as usual, then run the hooks task:
module.exports = function(grunt) {
  pkg: grunt.file.readJSON('package.json'),
  grunt.initConfig({ /* ... */ });

  grunt.loadNpmTasks('web-component-tester');
  grunt.loadNpmTasks('grunt-contrib-watch');
  grunt.loadNpmTasks('grunt-notify');
  grunt.task.run('notify_hooks');
};
That is all that is needed, but I like to specify two configuration options: one to send notifications when the tests succeed (only failure messages are displayed by default) and one to display the message for 8 seconds (instead of the default 3):
module.exports = function(grunt) {
  pkg: grunt.file.readJSON('package.json'),
  grunt.initConfig({
    'wct-test': { /* ... */ },

    watch: { /* ... */ },

    notify_hooks: {
      options: {
        success: true,
        duration: 8
      }
    }
  });

  grunt.loadNpmTasks('web-component-tester');
  grunt.loadNpmTasks('grunt-contrib-watch');
  grunt.loadNpmTasks('grunt-notify');
  grunt.task.run('notify_hooks');
};
With that, I see a nice success message when I start Grunt:



And, if I break something, I see:



I might like to see the actual failure message in there, but that is a minor quibble (and a byproduct of WCT since I saw just the one line in Gulp as well). That aside, this is exactly what I want in my test setup and should serve nicely for actual development of Polymer code—something that I will put to the test over the next few days.


Day #15


2 comments:

  1. Great article! Had a question regarding the web-component-tester :) I'm currently testing a web browser extension that's built with polymer elements, do you know if the web-component-tester is able to install browser extensions for testing and if wct is able to go to a specific website/url rather than a local or shared server?

    ReplyDelete