Send to Kindle

Sunday, December 28, 2014

A Solution for Karma Testing Polymer on IE (10 & 11)


I coded in WordPad yesterday. 15 years of successfully repressing that sensation and now I fear I may never recover.

Still, I successfully got Karma & Jasmine to test Polymer code with Internet Explorer yesterday. So it was worth it. Kinda.

Regardless, I am happy to have gotten testing of Polymer on IE working, but I still have a few outstanding questions that I'd like answered before moving on to less traumatizing subjects. My IE Polymer tests work with karma-ie-launcher and IE10. I would like to be able to run karma-webdriver-launcher and use it on both IE10 and IE11.

Switching to karam-webdriver-launcher would mean no more WordPad coding (since the code resides on the host machine, not the guest VM), so I will start there. I already have Karma configuration in place that should work:
module.exports = function(config) {
  config.set({
    // ...
    // Use IP so Windows guest VM can connect to host Karma server
    hostname: '192.168.1.129',
    customLaunchers: {
      'IE10': {
        base: 'WebDriver',
        config: {
          hostname: 'localhost',
          port: 4410
        },
        browserName: 'internet explorer',
        name: 'Karma'
      }
    },
    browsers: ['IE10']
  });
};
I was able to use this to connect to webdriver on Windows the other day, I just could not get the tests to pass. Without changing the tests, I still get very unhelpful failures:
$ karma start --single-run 
INFO [karma]: Karma v0.12.28 server started at http://192.168.1.129:9876/
INFO [launcher]: Starting browser internet explorer via Remote WebDriver
INFO [IE 10.0.0 (Windows 8)]: Connected on socket WUsG_gAEAHJFhm1lLF0G with id 61543069
IE 10.0.0 (Windows 8) ERROR
  Object doesn't support property or method 'indexOf'
  at /home/chris/repos/polymer-book/play/plain_forms/js/node_modules/karma-jasmine/lib/jasmine.js:1759
IE 10.0.0 (Windows 8): Executed 1 of 3 ERROR (0.645 secs / 0.547 secs)
This indexOf failure seems to come from calling a “contains” matcher in my tests on undefined values. As I found yesterday, that is more of a symptom of larger Karma issues than individual test failures.

So I try yesterday's wait-a-browser-event-loop / set-timeout-zero solution in my tests:
  describe('properties', function(){
    it('updates value when internal state changes', function(done) {
      el.model.firstHalfToppings.push('pepperoni');
      el.async(function(){
        setTimeout(function(){
          expect(el.value).toContain('pepperoni');
          done();
        }, 0);
      });
    });
  });
The el here is my <x-pizza> pizza building Polymer element. The asyc() method from Polymer accepts a callback that will be invoked after Polymer has updated the UI and all bound variables. That works on its own in Chrome and Firefox, but, as I found yesterday, Polymer's IE implementation seems to have a bug that requires an additional browser event loop before everything is ready.

And, with that set-timeout-zero, I have my IE WebDriver tests passing:
$ karma start --single-run 
INFO [karma]: Karma v0.12.28 server started at http://192.168.1.129:9876/
INFO [launcher]: Starting browser internet explorer via Remote WebDriver
INFO [IE 10.0.0 (Windows 8)]: Connected on socket oggMD2c6Q8c4-xq-Lk_- with id 68843724
IE 10.0.0 (Windows 8): Executed 3 of 3 SUCCESS (0.54 secs / 0.543 secs)
So what about IE11? Things were even worse in that VM because I have yet to get the console in the Web Developer Tools to start successfully, making troubleshooting next to impossible. Perhaps the set-timeout-zero fix works there as well?

In the Windows VM, I fire up a good old command prompt (I can't believe that it hasn't changed in 10+ years). In there, I start WebDriver via the webdriver node.js package:
C:\Users\IEUser>webdriver-manager start --seleniumPort 4411
I run the IE10 WebDriver on port 4410 and the IE11 WebDriver on 4411. So I update my karma.conf.js accordingly:
module.exports = function(config) {
  config.set({
    // ...
    // Use IP so Windows guest VM can connect to host Karma server
    hostname: '192.168.1.129',

    customLaunchers: {
      'IE10': { /* ... */ },
      'IE11': {
        base: 'WebDriver',
        config: {
          hostname: 'localhost',
          port: 4411
        },
        browserName: 'internet explorer',
        name: 'Karma'
      }
    },

    browsers: ['IE11']
  });
};
Unfortunately, when I try to run the same Karma tests against IE11, I find:
$ karma start --single-run 
INFO [karma]: Karma v0.12.28 server started at http://192.168.1.129:9876/
INFO [launcher]: Starting browser Internet Explorer 11 via Remote WebDriver
INFO [IE 11.0.0 (Windows 7)]: Connected on socket xu5uT_EaFT6kfCVrTOpE with id 86153250
IE 11.0.0 (Windows 7) <x-pizza> properties updates value when internal state changes FAILED
        TypeError: Unable to get property 'firstHalfToppings' of undefined or null reference
           at Anonymous function (/home/chris/repos/polymer-book/play/plain_forms/js/test/XPizzaSpec.js:29:7)
IE 11.0.0 (Windows 7) <x-pizza> syncing <input> values updates the input FAILED
...
IE 11.0.0 (Windows 7): Executed 3 of 3 (2 FAILED) (0.4 secs / 0.396 secs)
INFO [WebDriver]: Killed Karma test.
For whatever reason, it seems that my Polymer element is taking more than the usual single event loop to register in IE11. A single event loop is all that is required in Chrome, Firefox, and IE10. But IE11 requires an additional 10 milliseconds before it is ready:
describe('<x-pizza>', function(){
  var el, container;

  beforeEach(function(done){
    container = document.createElement("div");
    container.innerHTML = '<x-pizza></x-pizza>';
    document.body.appendChild(container);
    el = document.querySelector('x-pizza');

    setTimeout(done, 10); // Delay for elements to register in Polymer
  });

  describe('properties', function(){
    it('updates value when internal state changes', function(done) {
      // Same async + set-timeout-zero test here...
    });
  });
});
With that, I have my tests passing on IE11:
$ karma start --single-run 
INFO [karma]: Karma v0.12.28 server started at http://192.168.1.129:9876/
INFO [launcher]: Starting browser Internet Explorer 11 via Remote WebDriver
INFO [IE 11.0.0 (Windows 7)]: Connected on socket sQqPBHAEvsL1ujLtULSD with id 7115351
IE 11.0.0 (Windows 7): Executed 3 of 3 SUCCESS (0.634 secs / 0.566 secs)
And this seems to work reliably. I run through 10 single-run rounds of tests and all pass each time. If I drop down to 2 milliseconds, then tests start failing on occasion. If I really, really needed IE testing, I might bump these timeout delays for 50 or 100 milliseconds. For now, I'm just happy with a reliable local solution.

So it is ugly. It is rife with callbacks, set-timeout-zeros and other more arbitrary set-timeouts. But I have a working solution for testing Polymer on Internet Explorer. And I didn't even need to fire up WordPad.


Day #38

Saturday, December 27, 2014

Finally, Testing Polymer in Internet Explorer


I am not fond of Internet Explorer. The reasons are myriad, but not terribly helpful. Despite my dislike, I must admit that it tends to fail in ways that make sense. More often than not, when troubleshooting an “IE bug” I find myself wondering how code works in other browsers. Maybe that will wind up being the case with my Polymer testing woes.

I am running tests for my <x-pizza> Polymer element with Karma. The tests are written with the Jasmine testing library and look like:
describe('<x-pizza>', function(){
  var el, container;
  beforeEach(function(done){
    // Create <x-pizza> el and other setup...
  });
  // ...
  describe('properties', function(){
    it('updates value when internal state changes', function(done) {
      el.model.firstHalfToppings.push('pepperoni');
      el.async(function(){
        expect(el.value).toContain('pepperoni');
        done();
      });
    });
  });
});
The error for the past two nights (with karma-webdriver-launcher and karma-ie-launcher) occurs when I try to access the firstHalfToppings property of model. In the test, it is undefined:
C:\Users\IEUser\plain_old_forms>node node_modules\karma\bin\karma start --single-run --browsers IE
INFO [karma]: Karma v0.12.28 server started at http://localhost:9876/
INFO [launcher]: Starting browser IE
INFO [IE 11.0.0 (Windows 7)]: Connected on socket qNf8JVIEqjDNilL7AQFH with id 9528228
IE 11.0.0 (Windows 7) <x-pizza> properties updates value when internal state changes FAILED
        TypeError: Unable to get property 'firstHalfToppings' of undefined or null reference
           at Anonymous function (C:/Users/IEUser/plain_old_forms/test/XPizzaSpec.js:29:7)
But if I fire this element up in a web page, that property is very definitely defined:



After much futzing, I eventually trace this not to my code, but to Polymer's async() method, which comes on the line after the failure message. Things got so wonky that failures in previous tests would break other tests in non-helpful ways. Only after running single tests — with Karma/Jasmine's ddescribe() / iit() — was I able to identify the culprit as async(), which my test uses:
    it('updates value when internal state changes', function(done) {
      el.model.firstHalfToppings.push('pepperoni');
      el.async(function(){
        expect(el.value).toContain('pepperoni');
        done();
      });
    });
Polymer elements expose this method as a callback mechanism. The supplied callback (my test's assertion in this case), is only invoked once Polymer has updated the UI and all bound variables. This is ideal for testing because the test can be assured that all of the Polymer element's properties and visualizations have been updated—at least in Chrome and Firefox.

In Internet Explorer, it seems that nothing is updated until one more browser event loop. In other words, I have to wait for a setTimeout-0 in addition to the async() callback:
it('updates value when internal state changes', function(done) {
      el.model.firstHalfToppings.push('pepperoni');
      el.async(function(){
        setTimeout(function(){
          expect(el.value).toContain('pepperoni');
          done();
        }, 0);
      });
    });
This quite definitely fixes the test. I have other tests that magically pass with the addition of a setTimeout-0, but reliably fail without.

And once again, after some investigation of an “IE Bug,” I am left with a code error that is not IE's fault. This is almost certainly a Polymer bug. The entire point of async() is that the element is updated when the supplied callback is invoked. That I have to wait for another event loop on top of this seems quite wrong.


Day #37

Friday, December 26, 2014

Polymer and Karma-Ie-Launcher


Testing in Internet Explorer is a low priority for me. It always seems to be more effort than it is worth, given its dwindling market share. Still, I recognize that there is value and that some folks need it. So, even if I will not run automated tests for the code in Patterns in Polymer, I would like to be able to tell people that it does work.

Except that I was unable to get karma-webdriver-launcher to launch IE tests against Polymer last night. I suspect that the problems lie with WebDriver, so tonight, I try karma-ie-launcher instead.

First, I copy the code and tests onto my Windows VM. In there I add karma-ie-launcher to the list of NPM package.json dependencies:
{
  "name": "plain_old_forms",
  "devDependencies": {
    "grunt": "~0.4.0",
    "grunt-contrib-watch": "~0.5.0",
    "karma-jasmine": "~0.2.0",
    "karma-ie-launcher": ">0.0"
  }
}
After an npm install, I try to start Karma:
C:\Users\IEUser\plain_old_forms>karma start --single-run --browsers IE
'karma' is not recognized as an internal or external command,
operable program or batch file.
OK, so I install it globally:
C:\Users\IEUser\plain_old_forms>npm install -g karma
npm WARN optional dep failed, continuing fsevents@0.3.1

> ws@0.4.32 install C:\Users\IEUser\AppData\Roaming\npm\node_modules\karma\node_modules\socket.io\node_modules\socket.io-client\node_modules\ws
> (node-gyp rebuild 2> builderror.log) || (exit 0)

C:\Users\IEUser\AppData\Roaming\npm\node_modules\karma\node_modules\socket.io\node_modules\socket.io-client\node_modules\ws>node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild
karma@0.12.28 C:\Users\IEUser\AppData\Roaming\npm\node_modules\karma
├── ...
With that, when I try Karma now, I get:
C:\Users\IEUser\plain_old_forms>karma start --single-run --browsers IE
'karma' is not recognized as an internal or external command,
operable program or batch file.
Sigh. It looks like I need the full path instead:
C:\Users\IEUser\plain_old_forms>node node_modules\karma\bin\karma start --single-run --browsers IE
INFO [karma]: Karma v0.12.28 server started at http://localhost:9876/
INFO [launcher]: Starting browser IE
WARN [watcher]: Pattern "C:/Users/IEUser/plain_old_forms/bower_components/webcomponentsjs/webcomponents.js" does not match any file.
WARN [watcher]: Pattern "C:/Users/IEUser/plain_old_forms/bower_components/**" does not match any file.
INFO [IE 11.0.0 (Windows 7)]: Connected on socket _0g9fTtXw2PJOCjb5b2y with id 98303942
IE 11.0.0 (Windows 7) <x-pizza> element content has a shadow DOM FAILED
        ReferenceError: 'Polymer' is undefined
           at waitForPolymer (C:/Users/IEUser/plain_old_forms/test/PolymerSetup.js:19:5)
           at Anonymous function (C:/Users/IEUser/plain_old_forms/test/PolymerSetup.js:29:3)
...
IE 11.0.0 (Windows 7): Executed 3 of 3 (3 FAILED) ERROR (0.047 secs / 0.038 secs)
Oops! I neglected to bower install my client-side dependencies (this time I globally install from the start):
C:\Users\IEUser\plain_old_forms>npm install -g bower
C:\Users\IEUser\AppData\Roaming\npm\bower -> C:\Users\IEUser\AppData\Roaming\npm\node_modules\bower\bin\bower
bower@1.3.12 C:\Users\IEUser\AppData\Roaming\npm\node_modules\bower
└── ...
That does get installed in a useful location and it works… to a point:
C:\Users\IEUser\plain_old_forms>bower install
bower polymer#*                 ENOGIT git is not installed or not in the PATH
So it seems that I need Git installed to work with Polymer on Windows. This is already more trouble than it is worth to me, but I must see it through. I install Git the usual way and accept all of the default options but one:



This will allow bower to access Git when running from the command prompt:
C:\Users\IEUser\plain_old_forms>bower install
...
a-form-input#0.0.1 bower_components\a-form-input
└── polymer#0.5.2
polymer#0.5.2 bower_components\polymer
├── core-component-page#0.5.2
└── webcomponentsjs#0.5.2
core-component-page#0.5.2 bower_components\core-component-page
├── polymer#0.5.2
└── webcomponentsjs#0.5.2
webcomponentsjs#0.5.2 bower_components\webcomponentsjs
Yay!

Now, when I karma, I find:
C:\Users\IEUser\plain_old_forms>node node_modules\karma\bin\karma start --single-run --browsers IE
INFO [karma]: Karma v0.12.28 server started at http://localhost:9876/
INFO [launcher]: Starting browser IE
INFO [IE 11.0.0 (Windows 7)]: Connected on socket qNf8JVIEqjDNilL7AQFH with id 9528228
IE 11.0.0 (Windows 7) <x-pizza> properties updates value when internal state changes FAILED
        TypeError: Unable to get property 'firstHalfToppings' of undefined or null reference
           at Anonymous function (C:/Users/IEUser/plain_old_forms/test/XPizzaSpec.js:29:7)
IE 11.0.0 (Windows 7) <x-pizza> syncing <input> values updates the input FAILED
        TypeError: Unable to get property 'firstHalfToppings' of undefined or null reference
           at Anonymous function (C:/Users/IEUser/plain_old_forms/test/XPizzaSpec.js:42:7)
        TypeError: Unable to get property 'value' of undefined or null reference
           at Anonymous function (C:/Users/IEUser/plain_old_forms/test/XPizzaSpec.js:47:7)
IE 11.0.0 (Windows 7): Executed 3 of 3 (2 FAILED) (0.453 secs / 0.444 secs)
Dang it! Those are the same errors that I saw yesterday with WebDriver. So it seems that this is a problem with IE and Polymer rather than WebDriver. The failure is occurring when the test adds a topping to the first half of the pizza being built by the <x-pizza> Polymer element:
  describe('properties', function(){
    it('updates value when internal state changes', function(done) {
      el.model.firstHalfToppings.push('pepperoni');
      el.async(function(){
        expect(el.value).toContain('pepperoni');
        done();
      });
    });
  });
This works in Chrome and Firefox, but it seems that I need a different approach in Internet Explorer. Unfortunately, the web developer tools are still broken in my install so this is going to be tricky to troubleshoot.



Day #36

Thursday, December 25, 2014

Karma IE Testing of Polymer Elements with WebDriver


I think I am resigned to Karma as the best solution for Polymer—especially for the testing chapters in Patterns in Polymer. My experience with WebDriver, and Protractor in particular, made this a tougher decision than I expected. In the end, the combination of WebDriver's lack of shadow DOM support (kinda important with Polymer) and conceptual overload lead me to set Protractor aside. For now.

There are two things that I will miss from Protactor: wait-for-element baked in to element finders and easy Internet Explorer testing. There is not much to be done with async testing in Karma—that is more or less up to the testing library, most of which rely on done() callbacks. Those work, but are not as nice as Protractor's promise-based finders. I can live without them—especially in the book. But crazy as it might seem, I would like to be able to test on Internet Explorer.

In Protractor, I could establish a WebDriver instance in a Windows VM that my Protractor tests could drive from my Linux box. In Karma, I think I am stuck with karma-ie-launcher. There is not a ton of documentation on that project, but I assume that I would have to install Node.js, Karma, and my code on the Windows VM in order to make that work. I much prefer the code residing entirely on my machine with only a WebDriver instance running on the Windows VM.

Enter karma-webdriver-launcher. I add it to the list of NPM package dependencies:
{
  "name": "plain_old_forms",
  "devDependencies": {
    // ....
    "karma-webdriver-launcher": "~ 1.0.1"
  }
}
And then install:
$ npm install

karma-webdriver-launcher@1.0.1 node_modules/karma-webdriver-launcher
└── wd@0.2.8 (vargs@0.1.0, async@0.2.10, q@0.9.7, underscore.string@2.3.3, archiver@0.4.10, lodash@1.3.1, request@2.21.0)
To use in my Karma configuration, I add a custom launcher for IE11:
module.exports = function(config) {
  var webdriverConfig = {
    hostname: 'localhost',
    port: 4411
  };

  config.set({
    // ...
    customLaunchers: {
      'IE11': {
        base: 'WebDriver',
        config: webdriverConfig,
        browserName: 'internet explorer',
        name: 'Karma'
      }
    },

    browsers: ['IE11']
  });
};
I already have WebDriver installed on my Windows VM via the webdriver Node.js package. I start it with:
C:\Users\IEUser>webdriver-manager start --seleniumPort=4411
(I also have port forwarding to 4411 in place)

Then I run the tests from my Linux box, or I try. Instead of successful or failing tests, I see an error:
$ karma start --single-run
INFO [karma]: Karma v0.12.28 server started at http://localhost:9876/
...
INFO [launcher]: Trying to start internet explorer via Remote WebDriver again (2/2).
WARN [launcher]: internet explorer via Remote WebDriver have not captured in 60000 ms, killing.
INFO [WebDriver]: Killed Karma test.
ERROR [launcher]: internet explorer via Remote WebDriver failed 2 times (timeout). Giving up.
On the Windows side, I see that I am unable to connect to the Karma server on port 9876:



I was unaware of this, but it is possible to set the hostname for the Karma web server in the configuration file. So I add the IP address of my Linux box:
  config.set({
    // ...
    hostname: '192.168.1.129',

    customLaunchers: {
      'IE11': {
        base: 'WebDriver',
        config: webdriverConfig,
        browserName: 'internet explorer',
        name: 'Karma'
      }
    },
    browsers: ['IE11']
  });
And that actually works:
$ karma start --single-run
INFO [karma]: Karma v0.12.28 server started at http://192.168.1.129:9876/
INFO [launcher]: Starting browser internet explorer via Remote WebDriver
INFO [IE 11.0.0 (Windows 7)]: Connected on socket JiWBPa2Dzoq-fiGXsSvi with id 96899745
IE 11.0.0 (Windows 7) <x-pizza> properties updates value when internal state changes FAILED
        TypeError: Unable to get property 'firstHalfToppings' of undefined or null reference
           at Anonymous function (/home/chris/repos/polymer-book/play/plain_forms/js/test/XPizzaSpec.js:29:7)
IE 11.0.0 (Windows 7) <x-pizza> syncing <input> values updates the input FAILED
        TypeError: Unable to get property 'firstHalfToppings' of undefined or null reference
           at Anonymous function (/home/chris/repos/polymer-book/play/plain_forms/js/test/XPizzaSpec.js:42:7)
        TypeError: Unable to get property 'value' of undefined or null reference
           at Anonymous function (/home/chris/repos/polymer-book/play/plain_forms/js/test/XPizzaSpec.js:47:7)
IE 11.0.0 (Windows 7): Executed 3 of 3 (2 FAILED) (0.415 secs / 0.409 secs)
INFO [WebDriver]: Killed Karma test.
Well, it kind of works.

The connection is made and one of the test even passes. The two failing tests seem an awful lot like WebDriver and Polymer issues that I have seen over the past two weeks. Satisfied that I can connect, I will call it a night here. I may investigate the failures tomorrow.


Day #35

Wednesday, December 24, 2014

TDDing New Polymer Element Features with Protractor


Even after a week or so of working with Protractor as testing solution for Polymer, I still do not know if I have a good feel for recommending it as a solution. I begin to understand its strengths and weaknesses as a Polymer testing tool—its async support works brilliantly with Polymer while its complete lack of shadow DOM support is nearly a show stopper.

Nearly a show stopper, but when I can write tests like this:
describe('<x-pizza>', function(){
  beforeEach(function(){
    browser.get('http://localhost:8000');
  });
  it('updates value when internal state changes', function() {
    new XPizzaComponent().
      addFirstHalfTopping('pepperoni');

    expect($('x-pizza').getAttribute('value')).
      toMatch('pepperoni');
  });
});
How can I dismiss it? To be sure, the XPizzaComponent Page Object helps to mask some ugliness—but the same test in Karma has just as much shadow DOM ugliness, plus async ugliness.

I have already tried TDDing a bug fix with Protractor, which went reasonably well. So tonight I wonder what it is like TDDing a new feature? I have been playing with an early version of the <x-pizza> pizza building Polymer element as I explore these questions:



This version of the element has the exploration advantage of including some old-timey HTML form elements with which to play. The advantage here is that I made old-timey mistakes in the backing code. The bug that I TDD'd the other day resulted from adding unknown toppings to the pizza. I eliminated the bug, but it is still possible to click the “Add First Half Topping” button without choosing a topping. I would like to change that. So I write a Protractor test:
describe('<x-pizza>', function(){
  beforeEach(function(){
    browser.get('http://localhost:8000');
  });
  // ...
  it('does nothing when adding a topping w/o choosing first', function(){
    new XPizzaComponent().
      clickAddFirstHalfToppingButton();

    expect($('x-pizza').getAttribute('value')).
      toEqual('\n\n');
  });
});
I need to write the clickAddFirstHalfToppingButton() method for my page object. This is already done as part of the addFirstHalfTopping() method, but now I need it separate:
function XPizzaComponent() {
  this.selector = 'x-pizza';
}

XPizzaComponent.prototype = {
  // ...
  clickAddFirstHalfToppingButton: function() {
    browser.executeScript(function(selector){
      var el = document.querySelector(selector),
          button = el.$.firstHalf.querySelector("button");
      button.click();
    }, this.selector);
  }
};
The executeScript() method is hackery to work around Protractor's lack of shadow DOM support. It is not too horrible, but I am extremely tempted to factor the el assignment out into its own method, especially since both addFirstHalfTopping() and clickAddFirstHalfToppingButton() both assign it in the exact same way:
XPizzaComponent.prototype = {
  addFirstHalfTopping: function(topping) {
    browser.
      executeScript(function(selector, v){
        var el = document.querySelector(selector),
            select = el.$.firstHalf.querySelector("select");
      }, this.selector, topping);
  },

  clickAddFirstHalfToppingButton: function() {
    browser.
      executeScript(function(selector){
        var el = document.querySelector(selector),
            button = el.$.firstHalf.querySelector("button");
        // ...
      }, this.selector);
  }
};
The problem with trying to factor this out is that I am obtaining the element inside an anonymous function that is called by browser.executeScript() and the reason that I am doing this is because regular Protractor locators cannot find Polymer elements. The best that I can come up with is:
XPizzaComponent.prototype = {
  el: function(){
    return browser.executeScript(function(selector){
      return document.querySelector(selector);
    }, this.selector);
  },
  // ...
  clickAddFirstHalfToppingButton: function() {
    browser.
      executeScript(function(el){
        var button = el.$.firstHalf.querySelector("button");
        button.click();
      }, this.el());
  }
});
This is not too horrible, I suppose. But I am writing as much test code (and working as hard to write it) as I am with the actual code.

Regardless, I have a failing test:
$ protractor --verbose
Using the selenium server at http://localhost:4444/wd/hub
[launcher] Running 1 instances of WebDriver
<x-pizza>
  has a shadow DOM - pass
  updates value when internal state changes - pass
  syncs <input> values - pass
  does nothing when adding a topping w/o choosing first - fail

Failures:

  1) <x-pizza> does nothing when adding a topping w/o choosing first
   Message:
     Expected 'unknown

' to equal '

'.
   Stacktrace:
     Error: Failed expectation
    at [object Object].<anonymous> (/home/chris/repos/polymer-book/play/protractor/tests/XPizzaSpec.js:30:7)

Finished in 2.032 seconds
4 tests, 4 assertions, 1 failure
I can make that pass easily enough. My Polymer element's addFirstHalf() method simply needs a guard clause:
Polymer('x-pizza', {
  // ..
  addFirstHalf: function() {
    if (this.currentFirstHalf == '') return;
    this.model.firstHalfToppings.push(this.currentFirstHalf);
  },
  // ...
});
With that, I have my passing test:
$ protractor --verbose
Using the selenium server at http://localhost:4444/wd/hub
[launcher] Running 1 instances of WebDriver
<x-pizza>
  has a shadow DOM - pass
  updates value when internal state changes - pass
  syncs <input> values - pass
  does nothing when adding a topping w/o choosing first - pass

Finished in 2.012 seconds
4 tests, 4 assertions, 0 failures
And a nice new feature.

I normally do not care for refactoring tests, so the difficulty experienced tonight really should not bother me. But it does. The lack of shadow DOM support makes me think that I will need to write many helpers—if not an entire library—to make Protractor and Polymer really play nicely. If I struggle with the simple stuff like getting the Polymer element itself, this does not bode well for more in-depth testing adventures. I may play with this some more over the next few days, but I am leaning toward not using Protractor at this point.


Day #34

Tuesday, December 23, 2014

Can't Protractor Polymer on Internet Explorer


All right, let's see if I can figure out what went wrong while running my Polymer + Protractor tests in Internet Explorer. On the face of it, Polymer and Protractor are not an obvious combination since Polymer is a web component (not an application) and Protractor is designed to test single-page applications, specifically AngularJS. Also, WebDriver (on which Protractor is based) does not support shadow DOM. That said, there are some definite reasons to like the Polymer + Protractor combination. Plus, tests should work on all browsers, including Internet Explorer.

Emphasis on should.

I am unsure where I went wrong the other night, but trying to run this stuff on IE11 instead of 10 seemed a next logical step. So I retrace my steps from that post (and from the source post). On the windows virtual machine, I install node.js. I install Java. I open a command prompt and install protractor globally (npm install -g protractor).

Side note, I omitted installing the IE WebDriver in the previous post. When I try to start WebDriver without it, I see:
C:\Users\IEUser>webdriver-manager start --seleniumPort=4411
Selenium Standalone is not present. Install with webdriver-manager update --standalone
The fix is to tell the webdriver manager to install the IE driver:
C:\Users\IEUser>webdriver-manager update --ie
With that, I can start my WebDriver server:
C:\Users\IEUser>webdriver-manager start --seleniumPort=4411
20:27:18.602 INFO - Launching a standalone server
Setting system property webdriver.ie.driver to C:\Users\IEUser\AppData\Roaming\npm\node_modules\protractor\selenium\IEDriverServer.exe
20:27:18.894 INFO - Java: Oracle Corporation 25.25-b02
20:27:18.908 INFO - OS: Windows 7 6.1 x86
20:27:18.922 INFO - v2.44.0, with Core v2.44.0. Built from revision 76d78cf
20:27:19.145 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4411/wd/hub
...
20:27:19.353 INFO - Started SocketListener on 0.0.0.0:4411
Back on my Linux machine, I update the protractor.conf.js to point to this IE11 instance of WebDriver:
exports.config = {
  seleniumAddress: 'http://localhost:4411/wd/hub',
  capabilities: {
    'browserName': 'internet explorer',
    'platform': 'ANY',
    'version': '11'
  },
  // seleniumAddress: 'http://localhost:4444/wd/hub',
  specs: ['tests/XPizzaSpec.js'],
  onPrepare: function() {
    require('./tests/polymerBrowser.js');
    require('./tests/xPizzaComponent.js');
  }
};
And, I also need to point the specs to my IP address instead of localhost so that the IE VM can connect to it:
describe('', function(){
  beforeEach(function(){
    browser.get('http://192.168.1.129:8000');
    // browser.get('http://localhost:8000');
  });
  // Tests here...
});
Last, I add a port forwarding rule on the VM so that my Protractor tests on the Linux box can connect to that WebDriver address on post 4411 of localhost.

With that… I see the exact same behavior that I saw the other night with IE10:
$ protractor --verbose
Using the selenium server at http://localhost:4411/wd/hub
[launcher] Running 1 instances of WebDriver
<x-pizza>
  has a shadow DOM - pass
A Jasmine spec timed out. Resetting the WebDriver Control Flow.
The last active task was:
WebDriver.findElements(By.cssSelector("x-pizza"))
    at [object Object].webdriver.WebDriver.schedule (/home/chris/local/node-v0.10.20/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/webdriver.js:345:15)
    at [object Object].webdriver.WebDriver.findElements (/home/chris/local/node-v0.10.20/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/webdriver.js:934:17)
  ...
  updates value when internal state changes - fail
Sigh.

As I saw with IE, the first test, which asks the Polymer element to send back its shadow DOM innerHTML via a Protractor executeScript(), works. The second test, which adds pepperoni toppings to the <x-pizza> element then asserts on the value attribute, fails. What is weird about the failure is that I can see the pepperoni toppings added in WebDriver IE:



So the problem is not interacting with the Polymer element, it is querying the value property. Interacting with the element involved a bit of hackery since Protractor does not support the shadow DOM, but the hackery would seem to work. What does not work is the thing that should work without any hackery at all—querying element attributes.

Of course, the hackery could be partially to blame. This works in Chrome and Firefox, but that does not necessarily work in all browsers. To see if that is the case, I remove the page object that updates the pizza toppings:
describe('<x-pizza>', function(){
  beforeEach(function(){
    browser.get('http://192.168.1.129:8000');
    // browser.get('http://localhost:8000');
  });

  it('updates value when internal state changes', function() {
    // new XPizzaComponent().
    //   addFirstHalfTopping('pepperoni');

    expect($('x-pizza').getAttribute('value')).
      toMatch('pepperoni');
  });
});
The test should fail, but hopefully it will be an assertion failure, not a timeout.

Unfortunately, it has no effect.

I am at a loss at what to try next. The developer tools on my IE11 installation are useless (I mean more than normal—I cannot type in the console). I would like to be able to test against IE, but it is not that important to me. I am inclined to set this aside until inspiration (or a driver update) arrive. Until then, I will likely move onto to other topics.


Day #33

Monday, December 22, 2014

Fixing Polymer Bugs with Protractor


Today, I hope to get a feel for fixing bugs in Polymer elements by driving the fixes with Protractor.

A while back, I noticed a problem in my <x-pizza> pizza building element:



The problem occurs when adding non-standard toppings, which results in a JavaScript error. I actually noticed the problem while writing Protractor tests, so if I can fix it with another Protractor test, it would make for nice symmetry.

My Protractor test for adding valid toppings looks like:
describe('<x-pizza>', function(){
  beforeEach(function(){
    browser.get('http://localhost:8000');
  });
  it('updates value when internal state changes', function() {
    new XPizzaComponent().
      addFirstHalfTopping('pepperoni');

    expect($('x-pizza').getAttribute('value')).
      toMatch('pepperoni');
  });
});
That is nice and compact thanks to the Page Objects pattern—the XPizzaComponent in this case.

The page object makes it easy to write a new test that breaks things:
  it('adds "unknown" topping when invalid item selected', function() {
    new XPizzaComponent().
      addFirstHalfTopping('');

    expect($('x-pizza').getAttribute('value')).
      toMatch('unknown');
  });
Instead of choosing an option from the list, this should select the first item on the drop down list, which is blank. When I run the Protractor test, I get a failure, but not an error:
  1) <x-pizza> adds "unknown" topping when invalid item selected
   Message:
     Expected '

' to match 'unknown'.
   Stacktrace:
     Error: Failed expectation
    at [object Object].<anonymous> (/home/chris/repos/polymer-book/play/protractor/tests/XPizzaSpec.js:31:7)
If I add a browser.pause():
  it('adds "unknown" topping when invalid item selected', function() {
    new XPizzaComponent().
      addFirstHalfTopping('');

    browser.pause();

    expect($('x-pizza').getAttribute('value')).
      toMatch('unknown');
  });
Then I can see the error in the WebDriver console:



I find that I can also use this very verbose code to print out the console:
  it('adds "unknown" topping when invalid item selected', function() {
    new XPizzaComponent().
      addFirstHalfTopping('');

    browser.manage().logs().get('browser').then(function(browserLog) {
      console.log('log: ' + require('util').inspect(browserLog));
    });

    expect($('x-pizza').getAttribute('value')).
      toMatch('unknown');
  });
Regardless of how I find the error, it is fairly easy to fix it. The console in both cases reports the error to be at:
ReferenceError: _svgUnknown is not defined
    at x-pizza.Polymer._toppingMaker (http://localhost:8000/elements/x_pizza.js:94:12)
Looking at the _toppingMaker() method, I see right away that, unlike the SVG makers for know elements, I am trying to call _svgUnknown() as a function instead of a method:
  _toppingMaker: function(topping) {
    if (topping == 'pepperoni') return this._svgPepperoni;
    if (topping == 'sausage') return this._svgSausage;
    if (topping == 'green peppers') return this._svgGreenPepper;
    return _svgUnknown;
  },
And, indeed, converting that to a method resolves the bug:
  _toppingMaker: function(topping) {
    if (topping == 'pepperoni') return this._svgPepperoni;
    if (topping == 'sausage') return this._svgSausage;
    if (topping == 'green peppers') return this._svgGreenPepper;
    return this._svgUnknown;
  },
Just as importantly, I know have an e2e test that exercises this code, ensuring that I do not make a dumb mistake again:
<x-pizza>
  has a shadow DOM - pass
  updates value when internal state changes - pass
  syncs <input> values - pass
  adds "unknown" topping when invalid item selected - pass


Finished in 1.938 seconds
4 tests, 4 assertions, 0 failures
This might seem like a small thing for all that test code, but if I made that mistake once, I can make it again. Unless I have a test.


Day #32