Headless Polymer Testing with web-component-tester

The browser support in web-component-tester (WCT) is super cool. At first.

But after you run the test suite a few times—or automate it with file watchers—it quickly becomes super annoying. Any browsers that are configured to run tests pop up each time the test suite is run. And if the test suite is scheduled to run each time a change is made to the code, this behavior rapidly grows obnoxious.

So I would like to mitigate or prevent the browsers from popping when the tests are running. I start with mitigation since it is built into WCT via the persistent option. I mark the local WCT task that I am using as persistent in my Gruntfile.js:

module.exports = function(grunt) {
  pkg: grunt.file.readJSON('package.json'),
  grunt.initConfig({
    'wct-test': {
      local: {
        options: {remote: false, persistent: true}
      },
      // ...
    },
    // ...
  });
};

That does not quite work like I hope however. The browsers in which the tests are run are indeed persistent, but they are so persistent that they prevent the WCT task from completing. This leaves me with two windows that have the last state of the element being tested:



And my Grunt watch task is locked after beginning my tests in Chrome and Firefox:

$ grunt watch     
Running "notify_hooks" task

Running "watch" task
Waiting...
Running "notify_hooks" task

Running "wct-test:local" (wct-test) task
Starting Selenium server for local browsers
Selenium server running on port 47127
Web server running on port 43597 and serving from /home/chris/repos/polymer-book/play/web-component-tester
chrome 41                Beginning tests via http://localhost:43597/js/test/index.html?cli_browser_id=0
firefox 37               Beginning tests via http://localhost:43597/js/test/index.html?cli_browser_id=1

No matter, those persistent WCT grapes were sour anyway.

Instead I am really hoping to get Xvfb working. If I fire up a virtual window system in which the browsers can run the tests, then I never have to see browsers. That should be perfect (and much better than the persistent grapes).

In fact, thank to xvfb-run, this is trivial. All I have to is start my Grunt task with that wrapper script:

$ xvfb-run -a grunt watch
Running "notify_hooks" task

Running "watch" task
Waiting...
Running "notify_hooks" task

Running "wct-test:local" (wct-test) task
Starting Selenium server for local browsers
Selenium server running on port 42548
Web server running on port 35453 and serving from /home/chris/repos/polymer-book/play/web-component-tester
chrome 41                Beginning tests via http://localhost:35453/js/test/index.html?cli_browser_id=0
chrome 41                Tests passed
firefox 37               Beginning tests via http://localhost:35453/js/test/index.html?cli_browser_id=1
firefox 37               Tests passed
Test run ended with great success

chrome 41 (1/0/0)                       firefox 37 (1/0/0)                    

Done, without errors.
Completed in 7.450s at Thu Apr 02 2015 00:36:05 GMT-0400 (EDT) - Waiting..

So I have a passing test and I never have to see a single browser. If I make changes, the test suite is re-run in multiple browsers and I never see a single window.

I very much appreciate xvfb-run's -a option to automatically select a server port. If I have another session running on :99 (the default), xvfb-run will automatically chose an alternate port. Unfortunately, this has the side effect of not working with existing solutions for using Xvfb from within Grunt.

I can probably live with typing xvfb-run -a each time before starting the file-watching WCT test suite. But if I want to keep the command-line to a bare minimum, I can use grunt-shell:

$ npm install grunt-shell --save-dev

Then, in Gruntfile.js, I can load the shell tasks and configure my own:

module.exports = function(grunt) {
  pkg: grunt.file.readJSON('package.json'),
  grunt.initConfig({
    // ...
    shell: {
      test: {
        command: 'xvfb-run -a grunt watch'
      }
    },
    // ...
  });

  grunt.registerTask('default', ['shell:test']);

  grunt.loadNpmTasks('web-component-tester');
  grunt.loadNpmTasks('grunt-contrib-watch');
  grunt.loadNpmTasks('grunt-shell');
  grunt.loadNpmTasks('grunt-notify');
  grunt.task.run('notify_hooks');
};

If I make the new shell:test the default task (as above), I can start up a file-watching, headless WCT test suite with nothing more than:

$ grunt

Nice! Those are some tasty testing grapes.


Day #16

‹prev | My Chain | next›