Sunday, September 7, 2014

Failure: Expected Throw, Which Threw


I got started with the errata for Beta 1.0 of Dart for Hipsters er… Dart 1 for Everyone (not sure I'll ever get used to that). For the most part, it is pretty OK. There are issues, but nothing insurmountable. Except…

One of my tests is failing. Or maybe not.

I had all of my tests passing with Dart 1.5.8, but with 1.6.0, I find:
CONSOLE MESSAGE: FAIL: [bad invocation from noSuchMethod] cannot supply invocation mirror arguments
  Expected: throws 'Yikes _ajaxSave'
    Actual: <Closure: () => dynamic>
     Which: threw 'Yikes _ajaxSave")'

  package:unittest/src/simple_configuration.dart 119:7                         SimpleConfiguration.onExpectFailure
  package:unittest/src/simple_configuration.dart 15:28                         _ExpectFailureHandler.fail
  package:matcher/src/expect.dart 113:9                                        DefaultFailureHandler.failMatch
  package:matcher/src/expect.dart 73:29                                        expect
  ../varying_the_behavior/test/calling_methods_from_no_such_method.dart 38:13  run.<fn>.<fn>
  package:unittest/src/test_case.dart 102:37                                   _run.<fn>
Which is just weird, right? I expect this code throws 'Yikes _ajaxSave'. I find that, when run under test, it threw 'Yikes _ajaxSave". And this is counted as a test failure?!

I actually first noticed this error back when Dart 1.6 was still in beta. I spent a little time investigating any changes to throwing errors in 1.6, but could not find any. In the end, I thought to wait on the possibility that the language had a bug that would get sorted. That turns out not to be the case.

Since 1.6.0 arrived, I have been ignoring this as best I can so that I can focus on changes to the text of the book. It helps knowing that I will always have time to investigate in depth during these chain posts. So I get down to it.

The test is for some of Dart's dynamic code capabilities. In this test case, I have a method that reaches noSuchMethod() in a call. In this particular noSuchMethod(), I handle calls to #save, but any other calls should result in an error:
class HipsterModel {
  noSuchMethod(args) {
    if (args.memberName != #save) {
      var method_name = args.memberName.toString().
        replaceAll('Symbol("', '').
        replaceAll(new RegExp(r'@.+'), '');
      throw "Yikes ${method_name}";
    }
  }
}
The problem turns out not to be a bug, but a change to the way that symbols are converted to strings. Had I looked closer at the failure, I might have noticed that the message includes trailing double quotes. In other words, the output of toString() on a Symbol is different and I am no longer handling it properly.

I believe that the old format of Symbol toString() looked like Symbol("_ajaxSave@XXX"). I don't recall what information was embedded in the XXX part of the string, but the following code would try to strip away the data so that a “clean” failure could be thrown (and ultimately tested):
    if (args.memberName != #save) {
      var method_name = args.memberName.toString().
        replaceAll('Symbol("', '').
        replaceAll(new RegExp(r'@.+'), '');
      throw "Yikes ${method_name}";
    }
In Dart 1.6, calling toString() on a Symbol results in something like: Symbol("_ajaxSave"). I cannot argue with the change—this seems cleaner. And easier to convert into a string. In fact, I can use replaceAllMapped() to extract a regular expression placeholder to do it:
    if (args.memberName != #save) {
      var method_name = args.memberName.toString().
        replaceAllMapped(
          new RegExp(r'Symbol\("(.+)"\)'),
          (m)=> m[1]
        );
      throw "Yikes ${method_name}";
    }
The language designers make converting Symbol objects to strings intentionally awkward so that developers do not break symbol lookup when code is converted to JavaScript. If this were production code, I might just embed the full Symbol#toString() output in the error. Or allow Dart's normal noSuchMethod() handling to kick in. But, since this helps to test book code, I leave it.

With that, I have all my tests passing and dartanalyzer static analysis passing as well. So I can back to fixing errata.

Day #176

Saturday, September 6, 2014

Xfvb-run for Profitable JS Testing


Now that I am getting my Patterns in Polymer testing house back in order with eee-polymer-tests, it is time to revisit my whole project testing script.

It already works through each code chapter in the book:
#!/bin/bash

for X in book/code-js/parent*
  # Run all tests for a chapter X here...
do
For each chapter's tests, it already upgrades Bower packages (mostly to get the latest Polymer) and performs a single run of Karma:
    # ...
    # Update bower
    echo -n "Updating bower..."
    bower update --quiet
    echo " done."

    # Run the Karma tests
    karma start --single-run

    # Handle failure
    if [[ $? -ne 0 ]]; then
        echo
        echo "Some tests failed in $X."
        exit 1
    fi
    # ...
That still works, even with my updates to the testing setup, but it pops up a browser window to execute the tests. Instead, I would like to run this all under Xvfb. I don't know if I have a standard way of doing this, but I seem to think that I should start Xvfb with a display port (e.g. :99) then run everything with that same display port:
#!/bin/bash

Xvfb :99 2>&1 >/dev/null &
DISPLAY=:99

for X in book/code-js/parent*
do
  # Run all tests for a chapter X here...
do
That works, but it produces a lot of noise. Even redirecting STDOUT/STDIN to /dev/null like I am doing, I get:
Updating bower...Initializing built-in extension Generic Event Extension
Initializing built-in extension SHAPE
Initializing built-in extension MIT-SHM
Initializing built-in extension XInputExtension
Initializing built-in extension XTEST
Initializing built-in extension BIG-REQUESTS
Initializing built-in extension SYNC
Initializing built-in extension XKEYBOARD
...
Initializing built-in extension XVideo-MotionCompensation
Initializing built-in extension SELinux
Initializing built-in extension GLX
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
Even after individual runs, I get a lot of junk like:
GC: 4 objects of 16 bytes = 64 total bytes 0 private allocs
CURSOR: 1 objects of 8 bytes = 8 total bytes 0 private allocs
TOTAL: 5 objects, 72 bytes, 0 allocs
1 CURSORs still allocated at reset
CURSOR: 1 objects of 8 bytes = 8 total bytes 0 private allocs
TOTAL: 1 objects, 8 bytes, 0 allocs
1 CURSOR_BITSs still allocated at reset
TOTAL: 0 objects, 0 bytes, 0 allocs
It is just noise obscuring the important testing stuff. I do not really understand how redirecting STDOUT/STDIN is failing, but I just want it gone.

Eventually, I find xvfb-run (on my Ubuntu system). I thought this might be a shell interpreter—the kind that goes after the sh-bang at the top of my shell script. But it turns out to be even simpler. I remove the DISPLAY setting as well as the Xvfb call. Instead, this simply wraps the call to Karma (which starts the browser):
    # Run the Karma tests
    xvfb-run karma start --single-run
With that, the output from my whole-book test script is just Karma output:
$ ./scripts/test.sh
START: book/code-js/parent_child
Updating bower... done.
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 37.0.2062 (Linux)]: Connected on socket Pg5IMgnAOxMo5KIogEba with id 28554454
Chrome 37.0.2062 (Linux): Executed 3 of 3 SUCCESS (3.086 secs / 3.079 secs)
/home/chris/repos/polymer-book

START: book/code-js/parent_events
Updating bower... done.
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 37.0.2062 (Linux)]: Connected on socket VI7JUF1u0gF4z62ngGTV with id 68348849
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.078 secs / 0.074 secs)
/home/chris/repos/polymer-book
...
Success!
I can even take this a step further and switch Karma to log warnings only:
    # Run the Karma tests
    xvfb-run karma start --single-run --log-level warn
Which results in test output like:
$ /scripts/test.sh
[TEST] book/code-js/parent_child (Bower updated).
Chrome 37.0.2062 (Linux): Executed 3 of 3 SUCCESS (3.08 secs / 3.075 secs)

[TEST] book/code-js/parent_events (Bower updated).
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.067 secs / 0.061 secs)

Success!
Success indeed.


Day #175

Friday, September 5, 2014

Standardizing My Polymer Testing Strategy with Generators


Now that I have a nifty Polymer test generator in eee-polymer-tests, it is time to put it to use. The next oldest chapter in Patterns in Polymer is the parent-child chapter which explores communication strategies between parent and child custom elements.

It turns out that I already have tests for this chapter. Furthermore, it turns out that they are complete rubbish. I have tests that are waiting 1500ms:
  it('updates the parent', function() {
    waits(1500);
    runs(function(){
      expect(el.shadowRoot.textContent).toContain('Count: 1');
    });
  });
Hopefully I had a good reason for that, but it is still raises a red flag. You know what else raises a red flag? All tests failing:
karma start --single-run
Chrome 37.0.2062 (Linux) [parent/child] updates the parent FAILED
        TypeError: Cannot read property 'call' of undefined
            at HTMLElement.webkitCreateShadowRoot (/home/chris/repos/polymer-book/book/code-js/parent_child/scripts/polymer.min.js:34:7402)
            at Object.defineProperties.createShadowRoot.value [as createShadowRoot] (/home/chris/repos/polymer-book/book/code-js/parent_child/scripts/polymer.min.js:34:7641)
...
Chrome 37.0.2062 (Linux) [parent/child] updates the child FAILED
        TypeError: Cannot read property 'call' of undefined
            at HTMLElement.webkitCreateShadowRoot (/home/chris/repos/polymer-book/book/code-js/parent_child/scripts/polymer.min.js:34:7402)
            at Object.defineProperties.createShadowRoot.value [as createShadowRoot] (/home/chris/repos/polymer-book/book/code-js/parent_child/scripts/polymer.min.js:34:7641)
...
Chrome 37.0.2062 (Linux): Executed 2 of 2 (2 FAILED) ERROR (3.385 secs / 3.163 secs)
If I had to guess, I would say that the version of Polymer in use here is so old that it no longer works with Chrome. Rather than try to troubleshoot, I start fresh, which is what eee-polymer-tests does nicely. First, I delete everything related to old tests (they're still in git if I need them):
$ rm -rf node_modules karma.conf.js test bower.json bower_components
I add eee-polymer-tests to the list of NPM dependencies for this element's package.json:
{
  "name": "parent-events",
  "description": "Exploring communication between parent and child custom elements",
  "repository": {
    "type": "git",
    "url": "https://github.com/eee-c/patterns-in-polymer"
  },
  "devDependencies": {
    "grunt": "~0.4.1",
    "grunt-contrib-watch": "~0.5.3",
    "eee-polymer-tests": "eee-c/eee-polymer-tests"
  }
}
A quick npm install and I am ready to test... almost:
$ npm install
npm WARN package.json parent-events@ No README data
/
> eee-polymer-tests@0.0.1 postinstall /home/chris/repos/polymer-book/book/code-js/parent_child/node_modules/eee-polymer-tests
> ./generator.js

What is the name of the Polymer element being tested? x-parent

Generating test setup for: x-parent
[WARN] There is no elements subdirectory for Polymer elements. Tests will fail!
Done!

The Polymer test generator can be re-run with: ./node_modules/eee-polymer-tests/generator.js <x-element-name>
It turns out the code in this chapter is so old that, not only was the previous installed Polymer too old to work, I had not even adopted my convention of placing custom element code in the elements directory. Crazy!

Thanks to Bower's API, installing a fresh eee-polymer-tests will take care of installing the latest Polymer. I have to manually move the custom element definitions into the proper location. Once that is done, I still fail:
$ karma start --single-run
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 37.0.2062 (Linux)]: Connected on socket 6n0eubWSCoKotAYxEOhX with id 21476441
WARN [web-server]: 404: /base/elements/x-parent.html
Chrome 37.0.2062 (Linux) <x-parent> element content has a shadow DOM FAILED
        TypeError: undefined is not a function
            at waitForPolymer (/home/chris/repos/polymer-book/book/code-js/parent_child/test/PolymerSetup.js:20:15)
            at Object.<anonymous> (/home/chris/repos/polymer-book/book/code-js/parent_child/test/PolymerSetup.js:25:3)
        Expected null not to be null.
        Error: Expected null not to be null.
            at Object.<anonymous> (/home/chris/repos/polymer-book/book/code-js/parent_child/test/XParentSpec.js:23:33)
Chrome 37.0.2062 (Linux): Executed 1 of 1 (1 FAILED) ERROR (0.035 secs / 0.029 secs)
At first, I focus on the stacktrace. But eventually, I notice the warning message from the test server that there is no x-parent.html Polymer definition file. In fact, I had named it parent.html instead of x-parent.html. I do not know why I would have done this—the actual element name is <x-parent>.

Looks like I need to add another check to eee-polymer-tests' generator script. Once I resolve this by renaming the element, the dummy test passes. And, indeed the other tests (which did need the timeout after all), also pass when I add them back in:
karma start --single-run
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 37.0.2062 (Linux)]: Connected on socket bPbsoJmY6CEp5lr1HACY with id 22145708
Chrome 37.0.2062 (Linux): Executed 3 of 3 SUCCESS (3.08 secs / 3.074 secs)
Aside from some non-standard code locations, it seems that the overall approach in this chapter still works (phew!). Interestingly, I now have another chapter standardized on the same code structure, testing configuration, naming conventions, and the latest version of Polymer—all thanks to eee-polymer-tests and its generator. I have some updates to the chapter in order, but only filenames.


Day #174

Thursday, September 4, 2014

Generating Code from NPM that Installs Bower(!)


I have my eee-polymer-tests testing package for custom Polymer elements in pretty good shape. It will install and configure the necessary Karma configuration and skeleton tests to run a single, generated test on a Polymer element (which passes!). But...

My approach assumes two things: that the element definition is located in the elements subdirectory of the current package and that Bower is being used to install Polymer and its dependencies. Neither is a particularly poor assumption. The elements subdirectory is my personal preference, but I like it because it keeps my custom elements separate from any apparatus used to maintain the code. The Polymer project itself uses Bower for its elements, so I am OK sticking with it.

But, unlike all of the other checks that I have in eee-polymer-tests:
What is the name of the Polymer element being tested? click-sink

Generating test setup for: click-sink
[WARN] File exists: karma.conf.js. (use --force to overwrite)
[WARN] File exists: test/PolymerSetup.js. (use --force to overwrite)
[WARN] File exists: test/ClickSinkSpec.js. (use --force to overwrite)
I am not checking for the proper elements subdirectory, nor the Polymer + Bower configuration.

Testing for the existence of the elements directory is pretty straight-forward after my previous efforts:
function okElements() {
  if (!fs.existsSync('elements')) {
    var message = '[WARN] There is no elements subdirectory for Polymer elements. Tests will fail!';
    console.log(message.red);
    return false;
  }
  return true;
}
Checking for the existence of the bower.json is pretty much the same, except for the filename. I can even read the NPM module's package.json, which is what is running the generator in eee-polymer-tests, and write a corresponding bower.json if it does not already exist:
  var npmJson = JSOaN.parse(fs.readFileSync('package.json'));
  var bowerJson = {
    "name": npmJson.name,
    "version": "0.0.0",
    "description": npmJson.description,
    "ignore": [
      "**/.*",
      "node_modules",
      "bower_components",
      "test",
      "tests"
    ],
    "dependencies": {
      "polymer": "Polymer/polymer"
    }
  };
This gives me the brilliant idea to install Bower as part of eee-polymer-tests and ensure that Polymer is installed. Installing Bower is easy enough with NPM—it is, after all, the JavaScript package manager that is installed with the NPM JavaScript package manager. I add it to the list of eee-polymer-tests' dependencies:
{
  "name": "eee-polymer-tests",
  // ...
  "dependencies": {
    "colors": ">0.0",
    "minimist": ">0.0",
    "bower": ">0.0"
  },
  "peerDependencies": {
    "karma-jasmine": "~0.2.0",
    "karma-chrome-launcher": ">0.0"
  }
}
To actually install, it turns out that Bower has an API:
var bower = require('bower');
function generateBower() {
  // ...
  bower.commands
    .install(['Polymer/polymer'], { save: true }, { /* custom config */ })
    .on('end', function (installed) {
        console.log(installed);
     });
}
Which works quite as expected. Well, the installed object that is logged is just an empty object literal, but it does save changes to the bower.json file that is currently installed.

As cool as that is, I lean toward not using it. I have no idea if any besides myself will have use for eee-polymer-tests. Generating a simple test structure with it is one thing (and will be quite helpful), but I already install bower out of habit. I will likely file this under “good to know,” but focus elsewhere. Like actually writing some new tests tomorrow.

Update: Ah, the heck with it. I added bower generation after all: bfe13c3.

Day #173

Wednesday, September 3, 2014

Debugging Transient Test Failures with Bash One Liners


I confess that I am rather pleased with myself. I almost certainly messed something up, so it's not like I am going to get a big head or anything. But, for the moment, I am pleased with myself.

The new eee-polymer-tests generator for Karma / Jasmine testing of Polymer elements works pretty darn well. Even if I start a Polymer project with no tests:
$ rm -rf node_modules karma.conf.js test
Making the project dependent on eee-polymer-tests in an NPM package.json:
{
  "name": "parent-events",
  "devDependencies": {
    "eee-polymer-tests": "eee-c/eee-polymer-tests"
  }
}
Will install Karma, Jasmine and generate tests:
$ npm install
|
> eee-polymer-tests@0.0.1 postinstall /home/chris/repos/polymer-book/book/code-js/parent_events/node_modules/eee-polymer-tests
> ./generator.js

What is the name of the Polymer element being tested? click-sink

Generating test setup for: click-sink
Done!
Even better, the generated test, albeit quite simple, actually works without making any changes:
karma start --single-run
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 37.0.2062 (Linux)]: Connected on socket ciIVgf_I_GJhvyvorEtq with id 85510144
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.049 secs / 0.043 secs)
Well… it works most of the time. Every now and then, I get a failure like:
karma start --single-run
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 37.0.2062 (Linux)]: Connected on socket -cnZn5zrZF_OLuYjrF2l with id 31182079
Chrome 37.0.2062 (Linux) ERROR
  Uncaught TypeError: Cannot read property 'register' of undefined
  at /home/chris/repos/polymer-book/book/code-js/parent_events/bower_components/platform/platform.js:15
Oh, how I hate transient errors like this.

I get a failure like the one above roughly one out of every fives runs. To be sure that I have solved the problem, I am probably going to have to run the test file 10 times before I have any confidence. Fortunately, there are Bash one-liners for just this kind of thing:
$ for i in {1..10}; do karma start --single-run; done
Interestingly—and hopefully something of a clue—this passes for my hand-written specs (all two tests of them):
$ for i in {1..10}; do karma start --single-run; done
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.083 secs / 0.075 secs)
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.071 secs / 0.065 secs)
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.071 secs / 0.066 secs)
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.073 secs / 0.066 secs)
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.076 secs / 0.071 secs)
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.076 secs / 0.073 secs)
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.088 secs / 0.08 secs)
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.062 secs / 0.055 secs)
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.068 secs / 0.063 secs)
Chrome 37.0.2062 (Linux): Executed 2 of 2 SUCCESS (0.066 secs / 0.062 secs)
Perhaps this has something to do with the number of tests? To test that theory, I comment out one test and re-run the remaining test 10 times:
$ for i in {1..10}; do karma start --single-run; done
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.062 secs / 0.056 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.059 secs / 0.054 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.053 secs / 0.049 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.058 secs / 0.052 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.057 secs / 0.051 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.055 secs / 0.049 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.055 secs / 0.05 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.05 secs / 0.045 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.054 secs / 0.05 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.061 secs / 0.055 secs)
Nope. That's not it.

This suggests that either the problem is in the contents of my tests (the generated one is simpler than my actual hand-coded one that currently seems to work) or the problem has to do with the size of my test file (I only commented out the one test). It seems easier to delete the commented out test in my current spec as a next step. After doing so, I find:
$ for i in {1..10}; do karma start --single-run; done
Chrome 37.0.2062 (Linux) ERROR
  Uncaught TypeError: Cannot read property 'register' of undefined
  at /home/chris/repos/polymer-book/book/code-js/parent_events/bower_components/platform/platform.js:15

Chrome 37.0.2062 (Linux) ERROR
  Uncaught TypeError: Cannot read property 'resolveDom' of undefined
  at /home/chris/repos/polymer-book/book/code-js/parent_events/bower_components/polymer/polymer.js:14

Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.056 secs / 0.051 secs)
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.055 secs / 0.05 secs)
...
Bingo.

The most obvious solution—and possibly the one with which I am stuck—is to jam a bunch of comments in there. Heck I could even call it documentation. But why is this happening?

Update: Through process of elimination, I track this down to PolymerSetup.js, which does its level best to simulate a normal Polymer setup on a web page:
// 1. Load Polymer before any code that touches the DOM.
var script = document.createElement("script");
script.src = "/base/bower_components/platform/platform.js";
document.getElementsByTagName("head")[0].appendChild(script);

// 2. Load component(s)
var link = document.createElement("link");
link.rel = "import";
link.href = "/base/elements/click-sink.html";
document.getElementsByTagName("head")[0].appendChild(link);
The Polymer documentation is adamant that platform.js be the first script on a page—and this is probably why. And, even though it is the first script here and even though PolymerSetup.js is the first script in the Karma files list and even though it is the first file loaded (verified with --log-level debug on Karma), this is still not always the first file evaluated. And I think that is the cause. If my spec file is sufficiently small, it will be loaded and evaluated before this setup file is able to add platform.js in its proper place.

As I see it, I have three options:
  1. Move code that waits for polymer-ready out of PolymerSetup.js into the spec file (making the spec file large enough so that PolymerSetup.js loads and is evaluated first.
  2. Remove the platform.js inclusion from PolymerSetup.js and add it first to the files list in karma.conf.js
  3. Add junk data to be loaded before the spec files.
Crazy as it might seem, I may opt for #3. I have verified (via Bash for-loops) that all three options work. But the last one is probably the best Polymer solution.

The code that waits for polymer-ready feels like PolymerSetup.js so I would like to keep it there. The proper Karma solution would be #2, but I would end up with platform.js and the element being loaded from two different locations—and via two different mechanisms. The current PolymerSetup.js is not perfect, but it is as close to a normal Polymer solution as I can get (Karma makes it very hard to override the static page that holds the tests).

So adding a dummy file, with an explanation at the top, seems the best bet for my purposes.




Day #172

Tuesday, September 2, 2014

Better CLI in NPM Generators


Unsure of the best way to deal with existing files in eee-polymer-tests, I opt for a warn / force-overwrite approach. That is, when this NPM package attempts to overwrite existing Polymer test files of the same name, it should warn the developer of the problem, but make no changes. If a developer really wants to overwrite these files, she can either manually move them out of the way or use an option like --force.

So far, I have been able to build eee-polymer-tests with pure Node.js. Both of tonight's needs require their own packages. To warn users, I need to output the messages in yellow so that the problem is as obvious as possible. So far, my generators has accepted a single argument (the name of the Polymer element being tested), but I will need something a little more sophisticated to handle command-line options.

A quick search reveals colors and minimist (not minimalist) as my best bets for coloring strings on the command line and parsing the command line, respectively. So I add them to my package.json:
{
  "name": "eee-polymer-tests",
  "version": "0.0.1",
  "description": "Sane Polymer test setup (including generator)",
  "scripts": {
    "postinstall" : "./generator.js"
  },
  "homepage": "https://github.com/eee-c/eee-polymer-tests",
  "dependencies": {
    "colors": ">0.0",
    "minimist": ">0.0"
  }
}
In the generator script, I then require both:
var fs = require('fs');
var readline = require('readline');
var parseArgs = require('minimist');
require('colors');
For each of the files being generated, I add a conditional "ok-to-generate":
function generate() {
  ensureTestDirectory();
  if (okKarmaConf()) generateKarmaConf();
  if (okPolymerSetup()) generatePolymerSetup();
  if (okTestSkeleton()) generateTestSkeleton();
}
I factor out the check / warn logic common to each of these "ok-to-generate" function into a private _okOverwrite() function that checks for the existence of the file and colorizes a warning if the file already exists:
var KARMA_CONF = 'karma.conf.js';
// ...
function okKarmaConf() {
  return _okOverwrite(KARMA_CONF);
}

function _okOverwrite(filename) {
  if (!fs.existsSync(filename)) return true;

  var message = '[WARN] File exists: ' + filename + '.' +
                ' (use --force to overwrite)';
  console.log(message.yellow);
  return false;
}
That, combined with last night's readline generator works quite nicely:



The --force option is easy enough, thanks to minimist. I parse the command as:
function parseCommandLine() {
  if (process.argv.length <= 2) return;
  var argv = parseArgs(process.argv.slice(2), {
    'boolean': 'force'
  });
  if (argv._.length != 1) return;

  element = argv._[0];
  force = argv.force;
}
If there are no arguments, then the element variable is not set (and last night's readline solution will kick in). But, if there are arguments, I set the element value and, optionally, the force value.

Since I have already consolidated all of the ok-to-overwrite checking in a single place, I only have one place to update to accommodate the force value:
function _okOverwrite(filename) {
  if (!fs.existsSync(filename)) return true;

  if (force) {
    var message = 'Force overwrite: ' + filename + '.';
    console.log(message);
    return true;
  }

  var message = '[WARN] File exists: ' + filename + '.' +
                ' (use --force to overwrite)';
  console.log(message.yellow);
  return false;
}
With that, my generator still works as a post-install step and it also works directly from the command-line—optionally overwriting existing files if I really, really want to:



I am reasonably satisfied with that solution, so this may mark the end of initial development on eee-polymer-tests. There is still a minor issue with extremely small test files (like the generated spec) failing on rare occasions. I will have a look at that tomorrow. After that, it is time to finish off any remaining chapters in Patterns in Polymer that lack the kinds of tests that eee-polymer-tests can generate.

Day #171

Monday, September 1, 2014

Postinstall NPM Scripts with Output in the Working Directory


After last night's efforts on eee-polymer-tests, I can wipe my current Polymer (JS) testing install (NPM packages, tests, setup, and Karma config), install, generate new test setup, and run passing tests:
# Delete existing test install...
$ rm -rf karma.conf.js node_modules test

# Install eee-polymer-tests and dependencies..
$ npm install
karma-jasmine@0.2.2 node_modules/karma-jasmine
karma@0.12.23 node_modules/karma
karma-chrome-launcher@0.1.4 node_modules/karma-chrome-launcher
eee-polymer-tests@0.0.1 node_modules/eee-polymer-tests

# Generate new setup and sample test...
$ ./node_modules/eee-polymer-tests/generator.js click-sink

# Run tests...
$ karma start
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 37.0.2062 (Linux)]: Connected on socket ra4z9IfQR4BW8onHkNhy with id 82158706
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.046 secs / 0.041 secs)
That is going to be a huge help as I work through the remainder of Patterns in Polymer, adding tests to any book code that do not already have them. But there are still a couple of outstanding issues that I would like to knock off first. One of them is more an annoyance, but it will have to be dealt with: the small sample test tends to be so small that the Karma tests don't fully initialize Polymer before running, causing errors. Another problem is that I have not really accounted for the pre-existence of any of the generated files. The last problem is that I never seem to remember to run the generator manually.

I think I will work the problems in reverse order, since the last two are somewhat related by virtue of being NPM code. I can solve the last problem by making the script part of the install, which is done with one of the package lifecycle hooks in NPM. In this case, I make my eee-polymer-tests/generator.js script a post-install script:
{
  "name": "eee-polymer-tests",
  "version": "0.0.1",
  "description": "Sane Polymer test setup (including generator)",
  "scripts": {
    "postinstall" : "./generator.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  // ...
}
When I install, this script is run, but it fails.

It fails because it expects STDIN input to describe the element name:
var element = process.argv[2];
But there is no STDIN as part of a lifecycle script. I need a way to block for input. For that, I make use of the experimental Readline. Readline needs access to both input and output in order to read input and prompt for it:
var readline = require('readline');
var rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout
});
Using a readline instance is easy, thanks to the question() method:
rl.question("What is the name of the Polymer element being tested? ", function(answer) {
  element = answer.replace(/[<>]*/g, '');

  console.log("Generating test setup for: ", element);
  generate();
  console.log("Done!");

  rl.close();
});
Once I have my answer, I assign it to a global element variable for use in the templates being generated. After my generator does its thing, I tell the user that everything is ready and close the readline object.

And that works great. My generator is run after install and all of the code is generated correctly. Except...

It installs the generated code in the node_modules/eee-polymer-tests subdirectory instead of in the directory from which I called npm install. The whole point of this testing setup is to generate sane defaults and structure for my Polymer elements. So I need Jasmine tests and Karma configuration directly inside my Polymer elements—not inside the node_modules directory.

I am unsure if there is a proper way to get this working, but I solve it by checking the process.cwd() and then changing directories if this is being run from node_modules/eee-polymer-tests:
if (/node_modules\/eee-polymer-tests/.test(process.cwd())) {
  // Need to run the generator from the element package, not from the
  // installed node module
  process.chdir('../..');
}
And that does the trick. Now I can go from a clean Polymer install to passing tests with a single command:
$ rm -rf karma.conf.js node_modules test

$ npm install
> eee-polymer-tests@0.0.1 postinstall /home/chris/repos/polymer-book/book/code-js/parent_events/node_modules/eee-polymer-tests
> ./generator.js

What is the name of the Polymer element being tested? click-sink
Generating test setup for:  click-sink
Done!

karma-jasmine@0.2.2 node_modules/karma-jasmine
karma@0.12.23 node_modules/karma
karma-chrome-launcher@0.1.4 node_modules/karma-chrome-launcher
eee-polymer-tests@0.0.1 node_modules/eee-polymer-tests

$ karma start --single-run
INFO [karma]: Karma v0.12.23 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 37.0.2062 (Linux)]: Connected on socket SOOfrLj5phVcYX11idm1 with id 77662837
Chrome 37.0.2062 (Linux): Executed 1 of 1 SUCCESS (0.06 secs / 0.054 secs)
I still need to decide how to deal with pre-existing files of the same name as those being generated. I thought I had a good idea how to do that, but got mired in details tonight. Hopefully approaching it fresh will resolve things tomorrow. Then I need to see if I can resolve the spurious first run test failure in the generated code. That will no doubt be annoying and tedious since it does not happen all the time. If I am very lucky, I will have a brain wave that makes me understand why that happens and how to fix it. But I leave both tasks for another night.


Day #170