Forking Dart2JS in a Pub Transform (ICK)

My quest to generate a single JS file from a Polymer.dart source continues tonight with an attempt to get dart2js generated code into that single JS file.

After the previous two nights, I have the <polymer-element> template in said JS file (Polymer.dart puts it directly in the HTML by default) and various supporting libraries in that same JS file. But I lack the most important piece of this puzzle—the compiled backing class itself. The problem is that the dart2js transformer is run at the very end of the Dart Pub transform process.

The first thing that I try tonight is to place the $dart2js options before my custom, single JS transformer in my project's pubspec.yaml:

name: deployment_experiment
dependencies:
  polymer: any
dev_dependencies:
  scheduled_test: any
transformers:
- polymer:
    entry_points: web/index.html
- $dart2js:
    verbose: true
- deployment_experiment:
    entry_points: web/index.html

I have not looked into the $dart2js transformer code and the documentation never states this explicitly, but my sense is that the dart2js transformer is run last no matter what and that the $dart2js options are just that—options for that last transformer.

Still, best not to assume, so:

$ pub build
Building deployment_experiment.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................^C

I let that run for a good ten minutes with no discernable results. There is nothing happening to the filesystem:

➜  polymer-book git:(master) ✗ find play/deployment -mmin -60 -ls | grep -v node_modules | grep -v dart/packages
1188688    4 drwxr-xr-x  10 chris    chris        4096 Mar 31 22:17 play/deployment/dart
1196105    4 drwxr-xr-x   2 chris    chris        4096 Mar 31 21:58 play/deployment/dart/test
1186073    0 lrwxrwxrwx   1 chris    chris          11 Mar 31 21:58 play/deployment/dart/test/packages -> ../packages
1196106    4 drwxr-xr-x   2 chris    chris        4096 Mar 31 21:58 play/deployment/dart/web
1186074    0 lrwxrwxrwx   1 chris    chris          11 Mar 31 21:58 play/deployment/dart/web/packages -> ../packages
1188969    4 -rw-r--r--   1 chris    chris         349 Mar 31 22:10 play/deployment/dart/pubspec.yaml
1188960    4 -rw-r--r--   1 chris    chris        2203 Mar 31 21:58 play/deployment/dart/pubspec.lock
1188690    4 drwxr-xr-x   3 chris    chris        4096 Mar 31 22:00 play/deployment/dart/lib
1186103    4 -rw-r--r--   1 chris    chris        2240 Mar 31 22:00 play/deployment/dart/lib/transformer.dart
1196109    4 drwxr-xr-x   2 chris    chris        4096 Mar 31 22:10 play/deployment/dart/build

So I Ctrl-C the process.

If I move the $dart2js configuration to the end of pubspec.yaml, then the pub build works normally, albeit without the ability to transform dart2js output.

So ugh. It seems that I have do this the ugly way. The ugly way involves identifying the Polymer transform assets that get compiled into JavaScript by the default dart2js process at the end of pub build. Using that list, I then need to manually compiled these assets as part of my transform.

I identify the parts as part of the Transformer's determination of which elements are “primary” elements. These dart2js elements are not primary, but I squirrel them away in a list to be processed:

library single_file.transformer;

import 'dart:io';
import 'dart:async';
import 'package:barback/barback.dart';

class SingleJsPolymerTransformer extends Transformer {
  String entry_points;
  List<AssetId> bootstrappedPolymerElements = [];
  // ...
  Future<bool> isPrimary(Asset input) {
    if (input.id.path.contains('html_bootstrap.dart')) {
      bootstrappedPolymerElements.add(input.id);
    }

    if (entry_points == input.id.path) return new Future.value(true);
    return new Future.value(false);
  }
  // ...
}

With that, I can then run dart2js on the assets, but...

But I have to first write them out to the disk. The built-in dart2js transformer must have some kind of compile-from-asset capability. I will look into that at some point, but for tonight I content myself with ugly hacks like:

  buildSingleJavaScript(templateHtml, transform) {
    return transform.
      readInputAsString(bootstrappedPolymerElements.first).
      then((code) {
        new File('index.html_bootstrap.dart').
          openSync(mode: FileMode.WRITE).
          writeStringSync(code);

        var result = Process.runSync(
         'dart2js',
         ['-o', 'index.html_bootstrap.dart.js', 'index.html_bootstrap.dart']
        );
        // ...
      });
  }

Even that does not quite do the trick. It seems that there are secondary Polymer.dart elements that are required because I wind up with compilation errors along the lines of:

index.html_bootstrap.dart:7:8: Error: Can't read 'file:///home/chris/repos/polymer-book/play/deployment/dart/index.html.0.dart' (Error reading
 'index.html.0.dart' (OS Error: No such file or directory, errno = 2)).
import 'index.html.0.dart' as i2;
       ^^^^^^^^^^^^^^^^^^^
index.html_bootstrap.dart:15:3: Warning: Cannot resolve 'i2'.
  i2.main();
  ^^
Error: Compilation failed.

Bother.

I do think I am on the right track, but I call it a night at this point. I will pick back up fresh tomorrow either investigating how dart2js compiles assets or working through the remaining secondary Polymer.dart elements that are needed to compile this darn file.



Day #20

‹prev | My Chain | next›

Reading Files During Dart Pub Transform

After last night, I have my post-Polymer.dart pub transformer on the road to building single-file JavaScript from Dart Polymer elements.

This transformer needs to run after the built-in Polymer.dart transformer and takes identical configuration in the pubspec.yaml file:

name: deployment_experiment
dependencies:
  polymer: any
dev_dependencies:
  scheduled_test: any
transformers:
- polymer:
    entry_points: web/index.html
- deployment_experiment:
    entry_points: web/index.html

Where the built-in Polymer.dart transformer leaves the <polymer-element> templates directly in the entry_points HTML, my deployment experiment transformer moves the templates into a new JavaScript file responsible for adding the templates to the DOM.

To complete my transformer, I need to combine the remaining <script> tags from the Polymer.dart transformer into a single script source:

<!DOCTYPE html>
<html lang="en">
<head>
  <script src="/scripts/x_pizza_templates.js"></script>
  <script src="/scripts/shadow_dom.min.js"></script>
  <script src="/scripts/custom-elements.min.js"></script>
  <script src="/scripts/interop.js"></script>
  <script src="/scripts/index.html_bootstrap.dart.js"></script>
</head>
<body>
  <div class="container">
    <h1>Ye Olde Dart Pizza Shoppe</h1>
    <x-pizza></x-pizza>
  </div>
</body>
</html>

The problem I am faced with is that none of those files is included as an asset coming from the Polymer.dart transform. So I cannot put my (admittedly small) pub transformer knowledge to use here. Regardless of how all of that JavaScript is going to get into my single “deploy.js” file, I will need to do the work near the buildSingleJavaScript() method of my transformer:

import 'package:barback/barback.dart';

class SingleJsPolymerTransformer extends Transformer {
  // ...
  buildSingleJavaScript(templateHtml, transform) {
    var templateId = new AssetId(
      'deployment_experiment',
      'web/scripts/deploy.js'
    );
    transform.addOutput(
      new Asset.fromString(templateId, _templateJs(templateHtml))
    );
  }
  // ...
}

This method created a new asset ID to be added to the transform process and it then adds the asset it from a string (which is the crazy JavaScript string version of the <polymer-element> tags from last night).

The absolute dumbest way that I can think to add all of the other JavaScript source files to this asset is read each synchronously from the file system, concatenating each to last night's _templateJS():

  buildSingleJavaScript(templateHtml, transform) {
    var templateId = new AssetId(
      'deployment_experiment',
      'web/scripts/deploy.js'
    );

    var js = _templateJs(templateHtml) +
      new File('packages/shadow_dom/shadow_dom.min.js').readAsStringSync() +
      new File('packages/custom_element/custom-elements.min.js').readAsStringSync() +
      new File('packages/browser/interop.js').readAsStringSync();

    transform.addOutput(new Asset.fromString(templateId, js));
  }

That works perfectly. At least it works perfectly until I try to read the dart2js generated Polymer code:

  buildSingleJavaScript(templateHtml, transform) {
    var templateId = new AssetId(
      'deployment_experiment',
      'web/scripts/deploy.js'
    );

    var js = _templateJs(templateHtml) +
      new File('packages/shadow_dom/shadow_dom.min.js').readAsStringSync() +
      new File('packages/custom_element/custom-elements.min.js').readAsStringSync() +
      new File('packages/browser/interop.js').readAsStringSync() +
      new File('build/web/index.html_bootstrap.dart.js').readAsStringSync();

    transform.addOutput(new Asset.fromString(templateId, js));
  }

When I try to pub build with that in my transformer, I am greeted with:

➜  dart git:(master) ✗ pub build
Building deployment_experiment...
Build error:
Transform SingleJsPolymer on deployment_experiment|web/index.html threw error: Cannot open file, path = 'build/web/index.html_bootstrap.dart.js' (OS E
rror: No such file or directory, errno = 2)
file_impl.dart 558                                                            _File.throwIfError
file_impl.dart 411                                                            _File.openSync
file_impl.dart 456                                                            _File.readAsBytesSync
file_impl.dart 479                                                            _File.readAsStringSync
http://127.0.0.1:46880/packages/deployment_experiment/transformer.dart 54:74  SingleJsPolymerTransformer.buildSingleJavaScript
...

The problem, of course, is that build/web/index.html_bootstrap.dart.js does not exist yet. This file is generated when pub build calls dart2js on *.dart code after the various transforms are invoked. But I need the code during a transform, so am I out of luck?

In fact, I may be out of luck. About the only thing that I can think to try is forking out a separate dart2js process during the transform. That seems ugly, but there may be no other option save for giving up the idea of doing this as a transformer and sticking with shell scripts. As wrong as the fork-dart2js-during-transform feels, I will likely give that a try tomorrow—at least to see just how ugly it is. Unless someone has a better idea?


Day #19

‹prev | My Chain | next›

A Dart Pub Transform to Strip Polymer Elements

Today I start on a build script for a Polymer.dart element.

There is already a nice Dart build transformer that comes with the Polymer.dart package. It does a reasonable job of building a Polymer element into a web page for deployment to the modern web. I want to take it a step further. I would like my script to build the Polymer element for deployment to any page.

I have manually done just that over the past two days, using mostly command-line tools. I could probably work those command-lines into a Bash script without much difficulty, but I think it more useful to rework that effort in the form of another Dart transformer. This Dart transformer will (at least initially) use the results of the Polymer.dart transform, and transform them into a single script for deployment.

I am looking at this as a multi-day effort, so tonight I will extract the Polymer templates that the Polymer.dart transformer leaves in the web page that it builds. As I found two nights ago, it is possible to place them in a JavaScript file to be dynamically added to the DOM for ultimate use by the Polymer.dart platform.

For that, I need to write my own custom transformer. There is a nice article on the subject from the fine Dart folks, which saves me some work. I would like the transformer settings to follow the same convention as the original transformer—at least to start. So my pubspec.yaml becomes:

name: deployment_experiment
dependencies:
  polymer: any
dev_dependencies:
  scheduled_test: any
transformers:
- polymer:
    entry_points: web/index.html
- deployment_experiment:
    entry_points: web/index.html

The bulk of the work in this transformer, which I start in lib/transformer.dart, comes from the aforementioned article. This gives me a skeleton transformer of:

library single_file.transformer;

import 'dart:async';
import 'package:barback/barback.dart';

class SingleJsPolymerTransformer extends Transformer {
  String entry_points;

  SingleJsPolymerTransformer(this.entry_points);

  SingleJsPolymerTransformer.fromList(List a): this(a[0]);

  SingleJsPolymerTransformer.asPlugin(BarbackSettings settings)
    : this.fromList(_parseSettings(settings));

  Future<bool> isPrimary(Asset input) {
    if (entry_points == input.id.path) return new Future.value(true);
    return new Future.value(false);
  }

  Future apply(Transform transform) {
    // Need to do stuff here....
  }
}

List<String> _parseSettings(BarbackSettings settings) {
  var args = settings.configuration;
  return [ args['entry_points'] ];
}

Starting from that skeleton, I need to build the transform that will take the normal Polymer transform and convert everything into a single file. First up, my transformer needs to remove the <polymer-template> tags from the Polymer transformation step:

<!DOCTYPE html><html lang="en"><head>
<script src="/scripts/shadow_dom.min.js"></script>
<script src="/scripts/custom-elements.min.js"></script>
<script src="/scripts/interop.js"></script>
<script src="/scripts/index.html_bootstrap.dart.js"></script>
</head>
<body>
  <polymer-element name="x-pizza-toppings">
    <template><!-- ... --></template>
  </polymer-element>
  <polymer-element name="x-pizza">
    <template><!-- ... --></template>
  </polymer-element>
  <div class="container">
    <h1>Ye Olde Dart Pizza Shoppe</h1>
    <x-pizza></x-pizza>
  </div>
</body></html>

In pub transfomers, that means that I need to search and replace for text in a string, then update the asset in the transform. I accomplish that with:

class SingleJsPolymerTransformer extends Transformer {
  // ...
  Future apply(Transform transform) {
    var input = transform.primaryInput;
    var re = new RegExp(
      r'<polymer-element[\s\S]+/polymer-element>',
      multiLine: true
    );

    return transform.
      readInputAsString(input.id).
      then((html){
        var fixed = html.replaceAllMapped(
          re,
          (m) {
            return '';
          }
        );
        transform.addOutput(new Asset.fromString(input.id, fixed));
      });
  }
}

The hard part comes next. In addition to simply removing this text from an existing asset, I need to create a new asset (which requires a new, unique asset ID) and convert the <polymer-element> text into something that can be dynamically added via a JavaScript resource. Something like this ought to do the trick:

class SingleJsPolymerTransformer extends Transformer {
  // ...
  Future apply(Transform transform) {
    var input = transform.primaryInput;
    var re = new RegExp(
      r'<polymer-element[\s\S]+/polymer-element>',
      multiLine: true
    );

    return transform.
      readInputAsString(input.id).
      then((html){
        var polymer_template;
        var fixed = html.replaceAllMapped(
          re,
          (m) {
            polymer_template = m.group(0);
            return '';
          }
        );
        transform.addOutput(new Asset.fromString(input.id, fixed));

        var templateId = new AssetId(
          'deployment_experiment',
          'web/scripts/polymer_template.js'
        );
        transform.addOutput(
          new Asset.fromString(
            templateId,
            '''
var _html = '' +
${polymer_template.split('\n').map((s)=> "'" + s + "' +").join('\n')}
'';

window.addEventListener('DOMContentLoaded', function(){
  var container = document.createElement('div');
  container.innerHTML = _html;
  document.body.appendChild(container);
});
          '''
          )
        );
      });
  }
}

Inside the replaceAll, I squirrel away the <polymer-element> text in the polymer_template variable so that it can be used later. Then, I create a new web/scripts/polymer_template.js asset. Finally, I convert the <polymer-elements> into a bunch of concatenated JavaScript strings so that this will still work on the modern web (this is more or less what I did manually yesterday).

And, happily, this seems to work. If I use this generated JavaScript file in my build-generated code, then I still have a working Polymer. That seems a good stopping point for today. Up tomorrow, I need to combine this generated JavaScript code with the other JavaScript code that is normally generated in separate files by the normal Polymer transformer. My code is also a little messy, so some cleanup is in order. Hopefully this will prove relatively easy, but there is only one way to really know. Which I will find out tomorrow!


Day #18

‹prev | My Chain | next›

Deploying Polymer.dart in a Single, Compiled JavaScript File

Well that went better that I expected. Deploying Dart flavored Polymer elements, that is. So the question tonight it, am I overlooking something?

I've had troubles building Dart Polymer elements in the past, so I was trepidatious diving into it last night. But it went surprisingly well. At least some of my earlier trouble may have involved interoperating with JavaScript, but I still expected more trouble than I found last night. So let's see if I can cause trouble.

First up, let's see if I can do something about the sheer number of Dart compiled / supported JavaScript files that are necessary for my single <x-pizza> custom element:

<!DOCTYPE html>
<html lang="en">
<head>
  <script src="/scripts/x_pizza_templates.js"></script>
  <script src="/scripts/shadow_dom.min.js"></script>
  <script src="/scripts/custom-elements.min.js"></script>
  <script src="/scripts/interop.js"></script>
  <script src="/scripts/index.html_bootstrap.dart.js"></script>
</head>
<body>
  <div class="container">
    <h1>Ye Olde Dart Pizza Shoppe</h1>
    <x-pizza></x-pizza>
  </div>
</body>
</html>

These JavaScript files are already minified:

➜  scripts git:(master) ls -lh
total 800K
-rw-r----- 1 chris chris  11K Mar 27 23:14 custom-elements.min.js
-rw-r--r-- 1 chris chris 417K Mar 27 23:16 index.html_bootstrap.dart.js
-rw-r--r-- 1 chris chris 280K Mar 27 23:16 index.html_bootstrap.dart.js.map
-rw-r----- 1 chris chris  426 Mar 27 23:15 interop.js
-rw-r----- 1 chris chris  80K Mar 27 23:14 shadow_dom.min.js
-rw-r--r-- 1 chris chris 1.5K Mar 28 21:41 x_pizza_templates.js

Well, as minified as they get. The main index.html_bootstrap.dart.js file is definitely on the large side. There is not much that do about it as dart2js has done its best with a still early-in-development library. So, instead of worrying about making these any smaller, I hope to simply combine them into a single file:

➜  scripts git:(master) ✗ cat \
  x_pizza_templates.js \
  shadow_dom.min.js \
  custom-elements.min.js \
  interop.js \
  index.html_bootstrap.dart.js \
    > x_pizza.js

Well, of course I can combine them into a single file, but will it work for deployment purposes?

<!DOCTYPE html>
<html lang="en">
<head>
  <script src="/scripts/x_pizza.js"></script>
</head>
<body>
  <div class="container">
    <h1>Ye Olde Dart Pizza Shoppe</h1>
    <x-pizza></x-pizza>
  </div>
</body>
</html>

The answer, it would seem is a definite “yes”:



I am still able to build my pizzas with <x-pizza> in a non-Dart enabled Chrome (Firefox works as well) and am now using only a single, readily deployed <script> source to do so. Nice!

The other way that I thought to break this would be to include another compiled Dart script—event a simple one like:

import 'dart:html';
main(){
  query('h1').style.color = 'red';
}

After compiling that to JavaScript with dart2js, I pull the result into my deployment's script directory and try to use both this main() Dart script as well as the Dart Polymer element:

<!DOCTYPE html>
<html lang="en">
<head>
  <script src="/scripts/x_pizza.js"></script>
  <script src="/scripts/main.dart" type="application/dart"></script>
  <script src="/scripts/dart.js"></script>
</head>
<body>
  <div class="container">
    <h1>Ye Olde Dart Pizza Shoppe</h1>
    <x-pizza></x-pizza>
  </div>
</body>
</html>

And, happily, that works absolutely as desired:



So wow. This Polymer.dart deployment really works. Sure, I might like the size of the Polymer elements to be smaller, but the compile size will gradually improve as Dart and Polymer.dart evolve. But to have everything else working this well, this early is wonderful. I think tomorrow I will look to automate all of this.


Day #17

‹prev | My Chain | next›

Polymer Deployment as an Afterthought

I really don't care about deployment. Except, of course, when it's time to deploy.

At this point, I have written some 1500+ blog posts on a number of subjects. Nearly 200 have been on testing. Fewer than 30 have been on deployment. And that seems about right to me. Pushing out software, whether it is rubbish or not, should be facilitated by a language, library, or framework—and then automated away. Making the software unrubbishy is the trick.

Still, I'd like a better idea of how best to deploy Polymer. At this point, I am more concerned about deploying Dart-flavored Polymer elements than JavaScript ones, so I start there tonight.

There are two different ways that I can think to categorize Polymer.dart deployments. The first is when a Polymer element is part of another application. The second is when the Polymer element is intended to be used on a page independently of other libraries or applications. I think I have a handle on the former from my work integrating Angular.dart and Polymer.dart. That leaves me with the question of how to best prepare a single element, say my <x-pizza> pizza builder element, for ease of inclusion on a page.

With Dart Polymers, deployment currently begins and ends with Dart Pub—the packaging tool. For Polymers, the application's pubspec.yaml configuration file needs to specify web page entry points:

name: deployment_experiment
dependencies:
  polymer: any
dev_dependencies:
  scheduled_test: any
transformers:
- polymer:
    entry_points: web/index.html

A simple pub build then takes that entry point:

<!DOCTYPE html>
<html lang="en">
  <head>
    <!-- Load component(s) -->
    <link rel="import" href="packages/deployment_experiment/elements/x-pizza.html">
    <!-- Load Polymer -->
    <script type="application/dart">
      export 'package:polymer/init.dart';
    </script>
  </head>
  <body>
    <div class="container">
      <h1>Ye Olde Dart Pizza Shoppe</h1>
      <x-pizza></x-pizza>
    </div>
  </body>
</html>

And compiles the Dart code to JavaScript and prepares the HTML for deployment. Preparing the HTML mostly involves including the Polymer element templates directly in the HTML and sourcing the compiled JavaScript:

<!DOCTYPE html><html lang="en">
<head>
<script src="packages/shadow_dom/shadow_dom.debug.js"></script>
<script src="packages/custom_element/custom-elements.debug.js"></script>
<script src="packages/browser/interop.js"></script>

    <!-- Load component(s) -->
    <!-- Load Polymer -->
    <script src="index.html_bootstrap.dart.js"></script>
</head>
<body>
<polymer-element name="x-pizza-toppings">
  <template><!-- ... --></template>
</polymer-element>
<polymer-element name="x-pizza">
  <template><!-- ... --></template>
</polymer-element>

    <div class="container">
      <h1>Ye Olde Dart Pizza Shoppe</h1>
      <x-pizza></x-pizza>
    </div>

</body></html>

If I wanted to deploy most of this code to a CDN so that anyone could readily include <x-pizza> elements, I would copy the various libraries (although the “min” versions instead of the “debug” versions). To try this out, I create a deploy directory and copy said JavaScript libraries into it:

➜  dart git:(master) ✗ mkdir deploy
➜  dart git:(master) ✗ cd deploy 
➜  deploy git:(master) ✗ mkdir scripts
➜  deploy git:(master) ✗ cp ../packages/shadow_dom/shadow_dom.min.js scripts 
➜  deploy git:(master) ✗ cp ../packages/custom_element/custom-elements.min.js scripts 
➜  deploy git:(master) ✗ cp ../packages/browser/interop.js scripts 
➜  deploy git:(master) ✗ cp ../build/web/index.html_bootstrap.dart.js scripts 
➜  deploy git:(master) ✗ cp ../build/web/index.html_bootstrap.dart.js.map scripts

I then create a new HTML file referencing these scripts instead of the pub build package locations:

<!DOCTYPE html><html lang="en"><head>
<script src="/scripts/shadow_dom.min.js"></script>
<script src="/scripts/custom-elements.min.js"></script>
<script src="/scripts/interop.js"></script>
<script src="/scripts/index.html_bootstrap.dart.js"></script>
</head>
<body>
  <polymer-element name="x-pizza-toppings">
    <template><!-- ... --></template>
  </polymer-element>
  <polymer-element name="x-pizza">
    <template><!-- ... --></template>
  </polymer-element>
  <div class="container">
    <h1>Ye Olde Dart Pizza Shoppe</h1>
    <x-pizza></x-pizza>
  </div>
</body></html>

And it works just fine. The only real problem is that developers using this would have to include the <polymer-element> templates in addition to the <script> tags. It would feel more like a “regular” library if it were all <script> tags. So I move the templates out into a JavaScript file that builds them from JavaScript strings:

var xPizzaToppingsHtml = '' +
'<polymer-element name="x-pizza-toppings">' +
'  <template>' +
// ...
'  </template>' +
'</polymer-element>';

var xPizzaHtml = '' +
'<polymer-element name="x-pizza">' +
'  <template>' +
// ...
'  </template>' +
'</polymer-element>';

var container = document.createElement('div');
container.innerHTML = xPizzaToppingsHtml + xPizzaHtml;
document.body.appendChild(container);

With that, my deployment becomes:

<!DOCTYPE html>
<html lang="en">
<head>
  <script src="/scripts/shadow_dom.min.js"></script>
  <script src="/scripts/custom-elements.min.js"></script>
  <script src="/scripts/interop.js"></script>
  <script src="/scripts/index.html_bootstrap.dart.js"></script>
</head>
<body>
  <div class="container">
    <h1>Ye Olde Dart Pizza Shoppe</h1>
    <x-pizza></x-pizza>
  </div>
</body>
<script src="/scripts/x_pizza_templates.js"></script>
</html>

Ideally, I would replace the five different <script> tags with a single <script> source, but I leave that for another day. For now, I would just like to have a quick way to deploy a single element for reuse on any kind of web page. And I think I have that:



The JavaScript files are all coming from my deploy-ready scripts directory. The HTML is all regular HTML, save for my custom <x-pizza> Polymer element. I think my custom Dart Polymer element is ready for public use!

Well, maybe and maybe not. There are still a few questions to answer. I would like to know if this still works should another Dart application exist on the page. It would be nice to combine all of the scripts into a single source file. I would also like to be able to automate all of this.

That said, I am pleased to have gotten this working as well as I have in a single night. The promise of deployable Dart Polymer elements feels that much closer!


Day #16

‹prev | My Chain | next›

Polymer, Page Objects, and Jasmine 2.0

I am officially a Page Objects convert—especially when it comes to Polymer testing. After a bit of worry, I have Page Objects working in Dart even better that in JavaScript. This begs the question, can I apply some of the lessons learned from Dart back into JavaScript testing?

To answer that question, I am going to first make a slight change to my current Polymer JavaScript test setup. I have been using the Karma test runner, sticking with the default Jasmine test framework. Instead of the default Jasmine framework, I am going to switch to Jasmine 2.0, which (I think) has better support for asynchronous testing.

I start by adding the 2.0 Jasmine dependency to my package.json (the 2.0 dependency is karma-jasmine 0.2.0):

{
  "name": "model_example",
  "devDependencies": {
    "grunt": "~0.4.1",
    "grunt-contrib-watch": "~0.5.3",
    "karma-jasmine": "~0.2.0",
    "karma-chrome-launcher": ">0.0"
  }
}

After a quick npm install, I am ready to go.

First up, I need to fix my existing tests because waits(), waitsFor() and run() are gone in Jasmine 2.0:

Chrome 33.0.1750 (Linux) <x-pizza> adding a whole topping updates the pizza state accordingly FAILED
        ReferenceError: waitsFor is not defined
            at Object.<anonymous> (/home/chris/repos/polymer-book/play/model/js/test/PolymerSetup.js:14:3)
        ReferenceError: waitsFor is not defined
            at Object.<anonymous> (/home/chris/repos/polymer-book/play/model/js/test/XPizzaSpec.js:41:5)
        TypeError: Cannot read property 'wholeToppings' of undefined
            at Object.XPizzaComponent.addWholeTopping (/home/chris/repos/polymer-book/play/model/js/test/XPizzaSpec.js:11:29)
            at Object.<anonymous> (/home/chris/repos/polymer-book/play/model/js/test/XPizzaSpec.js:67:14)

That is fairly straight-forward. I need to replace waitsFor() with some equivalent that invokes Jasmine's done() callback. In this case, it is a nice improvement as this:

// Delay Jasmine specs until WebComponentsReady
beforeEach(function(){
  waitsFor(function(){
    if (typeof(CustomElements) == 'undefined') return false;
    return CustomElements.ready;
  });
});

Becomes simply:

beforeEach(function(done) {
  window.addEventListener('polymer-ready', done);
});

Learning a lesson from my Dart experience, I replace some ugly waitsFor() code with an async() call:

describe('<x-pizza>', function(){
  var container, xPizza;

  beforeEach(function(done){
    container = document.createElement("div");
    var el = document.createElement("x-pizza");
    container.appendChild(el);
    document.body.appendChild(container);

    xPizza = new XPizzaComponent(el);

    xPizza.el.async(done);
    // waitsFor(
    //   function() { return xPizza.currentPizzaStateDisplay() != ''; },
    //   "element upgraded",
    //   5000
    // );
  });
  // Tests here...
});

Just as with the Dart code, I realize that something is a little off here—I should not be directly accessing the el property of the XPizzaComponent Page Object. All interaction should go through the Page Object itself. The question is, how do I accomplish this without my beloved Futures?

Well, it ain't pretty, but enter callback hell:

describe('', function(){
  var container, xPizza;

  beforeEach(function(done){
    container = document.createElement("div");
    var el = document.createElement("x-pizza");
    container.appendChild(el);
    document.body.appendChild(container);

    xPizza = new XPizzaComponent(el);
    xPizza.flush(done);
  });
  // Tests go here...
});

As with the Dart code, the flush() method on the Page Object can invoke async() to get the Polymer platform to flush changes to observables, redraw things and then invoke the callback:

XPizzaComponent.prototype = {
  // ...
  flush: function(cb) {
    if (cb === undefined) cb = function(){};
    this.el.async(cb);
  }
};

Furthermore, I can invoke supplied callbacks via async() when supplied to Page Object interaction methods:

XPizzaComponent.prototype = {
  // ...
  addWholeTopping: function(v, cb) {
    // Choose options from select lists, click buttons, etc...
    return this.flush(cb);
  },

  flush: function(cb) {
    if (cb === undefined) cb = function(){};
    this.el.async(cb);
  }
};

So tests that use these page interaction methods can then be simplified to:

  describe('adding a whole topping', function(){
    beforeEach(function(done){
      xPizza.addWholeTopping('green peppers', done);
    });

    it('updates the pizza state accordingly', function(){
      var with_toppings = JSON.stringify({
        firstHalfToppings: [],
        secondHalfToppings: [],
        wholeToppings: ['green peppers']
      });

      expect(xPizza.currentPizzaStateDisplay()).
        toEqual(with_toppings);
    });
  });

To be sure, this is not as nice as the last night's Dart and Future based Page Object solution. Still, it is vastly superior (faster, more robust) than my previous take. Until Jasmine or some other testing framework supports promises, this will likely have to suffice. And for non-Dart code, it is not too bad.


Day #15

‹prev | My Chain | next›

OK Polymer, Why 500ms?

I hate to look a gift horse in the mouth, but...

After last night, I have my <x-pizza> pizza builder element well tested in both JavaScript and now Dart. Having struggled with the Dart testing in particular, it is extremely satisfying to run my tests and always see passing tests:

CONSOLE MESSAGE: unittest-suite-wait-for-done
CONSOLE MESSAGE: PASS: [defaults] it has no toppings
CONSOLE MESSAGE: PASS: [adding toppings] updates the pizza state accordingly
CONSOLE MESSAGE: 
CONSOLE MESSAGE: All 2 tests passed.
CONSOLE MESSAGE: unittest-suite-success
CONSOLE WARNING: line 213: PASS

And yet…

I only have this passing because I added a delay of 500ms to my test schedule:

    test('updates the pizza state accordingly', (){
      var toppings = JSON.encode({
        'firstHalfToppings': [],
        'secondHalfToppings': [],
        'wholeToppings': ['green peppers']
      });

      schedule(()=> xPizza.addWholeTopping('green peppers'));

      schedule(()=> new Future.delayed(new Duration(milliseconds: 500)));

      schedule((){
        expect(
          xPizza.currentPizzaStateDisplay,
          equals(toppings)
        );
      });
    });

Without that delay—or even if I try to decrease that delay—the test fails.

And so that delay just sits there, slowing down my test suite. Worse, it clutters up a pretty test, which would otherwise be super easy to read thanks to Page Objects (xPizza is a Page Object describing how to interact with the element) and scheduled_test, which enables scheduling asynchronous actions to run in sequence. And I cannot explain why this is even needed.

Well, I have some idea. Choosing a topping from the <x-pizza> pizza builder requires choosing an item from a <select> menu in a child Polymer element:



The <select> is in the “private” <x-pizza-toppings> element. This element fires a custom event to notify <x-pizza> that a change occurred:

@CustomTag('x-pizza-toppings')
class XPizzaToppings extends PolymerElement {
  // ...
  XPizzaToppings.created(): super.created() {
    model.
      changes.
      listen((_)=> fire('topping-change'));
  }
}

The parent <x-pizza> element listens for that event, updating the internal state accordingly:

@CustomTag('x-pizza')
class XPizza extends PolymerElement {
  XPizza.created(): super.created();
  enteredView() {
    super.enteredView();
    on['topping-change'].listen(updatePizzaState);
  }

  updatePizzaState([_]) {
    pizzaState = JSON.encode({
      'firstHalfToppings': $['firstHalfToppings'].model,
      'secondHalfToppings': $['secondHalfToppings'].model,
      'wholeToppings': $['wholeToppings'].model
    });
  }
}

My initial suspicion is that the various events and updates are taking too darn long for some unknown reason. To get to the bottom of this, I bring to bear my wide range of very sophisticated debugging techniques… OK, OK I scatter print() statements everywhere, but I started with the Chrome debugger for Dart, so that's got to count for something, right? Ahem.

Anyway, I eventually discern that, even with no scheduled delay at all, the internal state of <x-pizza> is being updated—it is just not updating the bound variables so that my test can see it.

What I need is a way to schedule a pause until the Polymer element has redrawn itself with the appropriate data—however long that might take. But that's just what James Hurford's suggestion from last night did! And, sure enough, if I replace the arbitrary wait for 500ms with an async() redraw action:

    test('updates the pizza state accordingly', (){
      var toppings = JSON.encode({ /* ... */ });

      schedule(()=> xPizza.addWholeTopping('green peppers'));

      // schedule(()=> new Future.delayed(new Duration(milliseconds: 500)));
      schedule((){
        var _completer = new Completer();
        xPizza.el.async((_)=> _completer.complete());
        return _completer.future;
      });

      schedule((){
        // Check expectations here ...
      });
    });

Then my tests pass. Every. Damn. Time.

So it seems that telling the Polymer element to “async” solves all my ills. This async() method tells the Polymer platform to flush(), which updates all observed variables including the one on which I am setting my test expectation. In other words, it makes complete sense that I would do this. The 500ms delay was ensuring that Platform.flush() was called and that my custom element was redrawn. Now I am just doing it more effectively.

This is so effective, that I ought to pull it back into the Page Objects pattern that I am using to interact with my Polymer element. The classic Page Objects pattern is for all methods to return a reference to the Page Object instance, which is what I have been doing with methods like addWholeTopping():

class XPizzaComponent {
  // ...
  XPizzaComponent addWholeTopping(String topping) {
    // Select items, click buttons, etc...
    return this;
  }
}

There is very little point to this in Dart, however. If I needed to chain method calls, I could simply use Dart's amazing method cascades. But, if I return a Future—say a Future that completes when async() finishes—then my Page Object becomes Polymer super-charged:

class XPizzaComponent {
  // ...
  Future addWholeTopping(String topping) {
    // Select items, click buttons, etc...
    return flush();
  }

  Future flush() {
    var _completer = new Completer();
    el.async((_)=> _completer.complete());
    return _completer.future;
  }
}

Why is that super-charged? Because scheduled_test schedules will wait until returned futures complete. This lets me write the entire test as:

    test('updates the pizza state accordingly', (){
      var toppings = JSON.encode({
        'firstHalfToppings': [],
        'secondHalfToppings': [],
        'wholeToppings': ['green peppers']
      });

      schedule(()=> xPizza.addWholeTopping('green peppers'));

      schedule((){
        expect(
          xPizza.currentPizzaStateDisplay,
          equals(toppings)
        );
      });
    });

The schedule that returns xPizza.addWholeTopping() now waits for async() finish. Once it does, the expectation passes. Every time.

So yay! Between scheduled_test and Page Objects, I have a nice, clean test suite. And, as much as I originally disliked the idea of Page Objects in my tests, I have to admit that they make a huge difference here. So much so that I believe I may need a new chapter in Patterns in Polymer!


Day #14

‹prev | My Chain | next›

Worky Scheduled Polymer Page Objects

Testing semi-complex Polymer.dart elements certainly has proved troublesome.

I am still working on my <x-pizza> pizza builder. The UI is not terribly complex:



The semi-complexity comes from the makeup of the Polymer element's <template>, which pulls in locally defined <x-pizza-toppings> sub-components:

<link rel="import" href="x-pizza-toppings.html">
<polymer-element name="x-pizza">
  <template>
    <h2>Build Your Pizza</h2>
    <pre id="state">{{pizzaState}}</pre>
    <x-pizza-toppings id="firstHalfToppings"
                      name="First Half Toppings"
                      ingredients="{{ingredients}}"></x-pizza-toppings>
    <x-pizza-toppings id="secondHalfToppings"
                      name="Second Half Toppings"
                      ingredients="{{ingredients}}"></x-pizza-toppings>
    <x-pizza-toppings id="wholeToppings"
                      name="Whole Toppings"
                      ingredients="{{ingredients}}"></x-pizza-toppings>
    <p class=help-block>
      Build Your Pizza!
    </p>
  </template>
  <script type="application/dart" src="x_pizza.dart"></script>
</polymer-element>

This is not too complex—just complicated enough to warrant the “semi-complex” label. And to give me fits testing.

Most of my fits are due to the asynchronous nature of Polymer elements—especially when sub-components come into play. I had hoped that relying on scheduled_test, which is a unittest replacement with nice asynchronous support, might solve my problems. But no.

James Hurford was kind enough to give me a hint about readying Polymer elements for testing. He suggested that, after the element was created:

  setUp((){
    schedule((){
      _el = createElement('<x-pizza></x-pizza>');
      document.body.append(_el);

      xPizza = new XPizzaComponent(_el);
    });
    // ...
  });

That I could schedule an async call which would fire after updating the UI. That seems like it might be right up my alley:

  setUp((){
    schedule((){ /* Create element... */ });

    schedule((){
      var _completer = new Completer();
      _el.async((_)=> _completer.complete());
      return _completer.future;
    });
  });

That actually helps quite a bit, but it does not solve all of my problems. It really seems as though Polymer is extremely sensitive to timing. I had accidentally left a delay in the my setup:

  setUp((){
    schedule((){ /* Create element... */ });

    schedule((){
      var _completer = new Completer();
      _el.async((_)=> _completer.complete());
      return _completer.future;
    });

    schedule(()=> new Future.delayed(new Duration(milliseconds: 750)));
  });

When removed, everything started working. I have no idea why that delay would prevent my tests from seeing changes in the Polymer, but that is how the failures would manifest—the original values would show rather than updates in response to page updates.

I also performed a bit of code cleanup in the Polymer code itself. Specifically, I found it best to listen for events in the enteredView() (will eventually be renamed attached()) callback:

@CustomTag('x-pizza')
class XPizza extends PolymerElement {
  // ...
  XPizza.created(): super.created();
  enteredView() {
    super.enteredView();

    on['topping-change'].listen((_){
      updatePizzaState();
    });
    updatePizzaState();
  }
  // ...
}

That code had been in the created() constructor, but it seemed more robust in enteredView(). Specifically, it worked with multiple elements and tests without random failures.

So it took a bit (and much thanks to James for the tip), but I finally have a robust test suite for this Polymer element.


Day #13

‹prev | My Chain | next›

Scheduled Polymer Page Objects Tests

Page Objects for testing Polymer seem to be working out for me nicely. What's not working out so well for me is asynchronous testing—with either the Dart or JavaScript versions of Polymer.

If anything, testing is a little easier in JavaScript, thanks to Jasmine's asynchronous helpers like wait() and run(). As odd as it may seem, I am struggling more with Dart testing than JavaScript testing. Thankfully, the fine Dart folks maintain scheduled_test for asynchronous testing in Dart. Let's see if that make my life any easier when testing Polymer code.

I start by switching my development dependencies from vanilla unittest to scheduled_test in my projects pubspec.yaml:

name: model_example
dependencies:
  polymer: any
dev_dependencies:
  scheduled_test: any
transformers:
- polymer:
    entry_points: web/index.html

A quick pub update and I am ready:

➜  dart git:(master) ✗ pub update
Resolving dependencies.....................
...
Downloading scheduled_test 0.10.1...
...
Changed 2 dependencies!

The resulting tests are much cleaner. That is not too surprising given my experience in the past with scheduled_test. For instance, adding a whole topping to my <x-pizza> pizza builder element is now just a series of schedules:

  group("[adding toppings]", (){
    test('updates the pizza state accordingly', (){
      var toppings = JSON.encode({
        'firstHalfToppings': [],
        'secondHalfToppings': [],
        'wholeToppings': ['green peppers']
      });

      schedule(()=> new Future.delayed(new Duration(milliseconds: 150)));

      schedule(()=> xPizza.addWholeTopping('green peppers'));

      schedule(()=> new Future.delayed(new Duration(milliseconds: 1500)));

      schedule((){
        expect(
          xPizza.currentPizzaStateDisplay,
          equals(toppings)
        );
      });
    });
  });

Some of those schedules are delays to allow the Polymer platform to do its thing. Others are the Page Objects interaction or interrogation.

But as nice as this is to read, it is not working any better for me. If anything, things are worse. No matter what delays I try, I am still getting test failures. Worse, the test failures seem to be more consistent. With the non-scheduled_test tests, I could reliably get the tests to pass under content_shell, even if they usually failed in Dartium. With scheduled_test, I now reliably get failures in both content_shell and Dartium.

Bother. I suspect my tests are trying to tell me something—especially now that I am (hopefully) being more strict with the order in which test code is evaluated. But I'll be darned if I can figure out what my tests are saying. Stumped, I call it a night here, but I think I need to give this another go tomorrow.


Day #12

‹prev | My Chain | next›

Refactoring using Page Objects in Tests

I am still looking for a good reason to hate Page Objects.

Page Objects are a pattern of testing that wraps an object around a page such that any action that a user might take (be it happy path or error generating) is represented as a method. Page Objects also expose getter methods to extract state information from the page for testing. Any action a user takes is represented by a method. Any test expectations should be made against getters. It sounds wonderful.

So why do I hate it? Mostly I hate indirection or anything that begins to suggest cleverness in testing. I like all my tests procedural and right up front. I would much rather copy and paste test code because, when the test inevitably fails, I hate tracing through clever testing code—it's bad enough tracing through clever application code.

All that said, I have found very little reason to dislike Page Objects when testing Polymer elements. They even seem to work quite nicely in both the JavaScript and Dart versions Polymer. Darn it.

So tonight, I am going to put Page Objects to a bit more stringent a test—I am going to refactor my Dart Polymer element. When I started on my pizza builder <x-pizza> component in JavaScript, I made the mistake of refactoring first and testing second. I did not make that same mistake with the Dart version. I wrote my tests last night and tonight I refactor.

The refactoring goes back to how the different toppings are added to the pizza builder:



Among some very excellent advice that I received on Patterns in Polymer was that the model in the Model Driven View code of <x-pizza> was overly complex. It was suggested that instead of placing the model in <x-pizza> itself, I should put the model into <x-pizza-topping> elements that are then created to be private child elements of <x-pizza>.

So let's see how that works with the Dart version of <x-pizza> and, more importantly, how Page Objects adapt to the change.

As I found with the JavaScript version of this Polymer element, the changes do make life easier for me. Mostly, it involved replacing a lot of template code in the element that looks like:

<polymer-element name="x-pizza">
  <template>
    <h2>Build Your Pizza</h2>
    <pre id="state">{{pizzaState}}</pre>
    <!-- first & second half toppings done here .. -->
    <p id="wholeToppings">
      <select class="form-control" value="{{currentWhole}}">
        <option>Choose an ingredient...</option>
        <option value="{{ingredient}}" template repeat="{{ingredient in ingredients}}">
          {{ingredient}}
        </option>
      </select>
      <button on-click="{{addWhole}}" type="button" class="btn btn-default">
        Add Whole Topping
      </button>
    </p>
  </template>
  <script type="application/dart" src="x_pizza.dart"></script>
</polymer-element>

With custom, private Polymer elements:

<link rel="import" href="x-pizza-toppings.html">
<polymer-element name="x-pizza">
  <template>
    <h2>Build Your Pizza</h2>
    <pre id="state">{{pizzaState}}</pre>
    <!-- first & second half toppings done here .. -->
    <x-pizza-toppings id="wholeToppings"
                      name="Whole Toppings"
                      ingredients="{{ingredients}}"></x-pizza-toppings>
    <p class=help-block>
      Build Your Pizza!
    </p>
  </template>
  <script type="application/dart" src="x_pizza.dart"></script>
</polymer-element>

Nice. That is easier to read and thus easier to maintain. It is a win all around. Except my Page Objects based test now fails:

FAIL: [adding toppings] updates the pizza state accordingly
  Expected: '{"firstHalfToppings":[],"secondHalfToppings":[],"wholeToppings":["green peppers"]}'
    Actual: '{"firstHalfToppings":[],"secondHalfToppings":[],"wholeToppings":[]}'
     Which: is different.
  Expected: ... oppings":["green pep ...
    Actual: ... oppings":[]}
                          ^
   Differ at offset 65

And here is where Page Objects really seems to help. To fix this, I only need update the addWholeTopping() method in my Page Objects wrapper around <x-pizza>. Instead of querying for the drop-down and button to add toppings, I can use Polymer's automatic node-finding:

class XPizzaComponent {
  PolymerElement el;
  XPizzaComponent(this.el);

  // Extract information from the page
  String get currentPizzaStateDisplay => el.$['state'].text;

  // Interact with the page
  XPizzaComponent addWholeTopping(String topping) {
    var toppings = el.$['wholeToppings'],
        select = toppings.$['ingredients'],
        button = toppings.$['add'];

    // Choosing an item from the drop-down and clicking the button
    // stay the same...

    return this;
  }
}

With that, I have my tests all passing again. I am not sure that qualifies as a real-life test of Page Objects because I only have one interaction method and one query method (which didn't change). Still, I am quite happy with the results here. I am very close to considering myself a Page Objects convert—at least with Polymer code.



Day #11

‹prev | My Chain | next›

Polymer Page Objects in Dart

I remain skeptical that Page Objects are a good idea for long term maintenance of a test suite. But after last night, they sure seem like they can come in handy.

So tonight, I give them a try in my Dart test suite for a simple <x-pizza> Polymer element:



I start a new test.dart with a Page Object wrapper for <x-pizza> elements. To begin with, I only need a getter that extracts the current pizza state from a pre#state element inside the Polymer element:

class XPizzaComponent {
  PolymerElement el;
  XPizzaComponent(this.el);
  String get currentPizzaStateDisplay => el.$['state'].text;
}

Then, I do the usual Polymer.dart test setup, calling initPolymer(), adding my <x-pizza> element to the test page, and finally creating a Page Objects instance:

main() {
  initPolymer();

  PolymerElement _el;
  XPizzaComponent xPizza;

  setUp((){
    _el = createElement('<x-pizza></x-pizza>');
    document.body.append(_el);

    xPizza = new XPizzaComponent(_el);
  });
  // ...
}

With that, I can write my first Page Object test as:

  group("[defaults]", (){
    test('it has no toppings', (){
      var no_toppings = JSON.encode({
        'firstHalfToppings': [],
        'secondHalfToppings': [],
        'wholeToppings': []
      });

      expect(
        xPizza.currentPizzaStateDisplay,
        no_toppings
      );
    });
  });

That fails because this is an older version of the Polymer element that is not JSON encoding the pizza state:

CONSOLE MESSAGE: FAIL: [defaults] it has no toppigns
  Expected: '{"firstHalfToppings":[],"secondHalfToppings":[],"wholeToppings":[]}'
    Actual: 'First Half: []
'   
    'Second Half: []
'   

To get the test passing, I update my Polymer class to properly JSON encode the pizza state:

import 'package:polymer/polymer.dart';
import 'dart:convert';

@CustomTag('x-pizza')
class XPizza extends PolymerElement {
  // ...
  Pizza model;
  @observable String pizzaState;

  XPizza.created(): super.created() {
    // ...
    updatePizzaState();
  }

  updatePizzaState([_]) {
    pizzaState = JSON.encode({
      'firstHalfToppings': model.firstHalfToppings,
      'secondHalfToppings': model.secondHalfToppings,
      'wholeToppings': model.wholeToppings
    });
  }
  // ...
}

Now comes the fun part.

I need to be able to select items from the <select> list in my Polymer element. That was a pain in the JavaScript version of this element. Thankfully, the Page Object version of the addWholeTopping() method in Dart is pretty straight-forward. At least, it is straight-forward having the JavaScript version to reference. The Dart version is:

class XPizzaComponent {
  PolymerElement el;
  XPizzaComponent(this.el);
  String get currentPizzaStateDisplay => el.$['state'].text;
  XPizzaComponent addWholeTopping(String topping) {
    var toppings = el.$['wholeToppings'],
        select = toppings.query('select'),
        button = toppings.query('button');

    var index = -1;
    for (var i=0; i<select.length; i++) {
      if (select.options[i].value == topping) index = i;
    }
    select.selectedIndex = index;

    var event = new Event.eventType('Event', 'change');
    select.dispatchEvent(event);

    button.click();

    return this;
  }
}

That is more or less copied-and-pasted from the JavaScript version of the Page Object. As in JavaScript, I find that I have to dispatch a change event to the <select> element. That aside, this method is straight procedural code selecting an item and clicking a button.

Armed with that, I can write the next test as:

  group("[adding toppings]", (){
    setUp((){
      var _completer = new Completer();
      Timer.run(_completer.complete);
      return _completer.future;
    });

    test('updates the pizza state accordingly', (){
      xPizza.addWholeTopping('green peppers');

      var toppings = JSON.encode({
        'firstHalfToppings': [],
        'secondHalfToppings': [],
        'wholeToppings': ['green peppers']
      });

      new Timer(
        new Duration(milliseconds: 100),
        expectAsync((){
          expect(
            xPizza.currentPizzaStateDisplay,
            toppings
          );
        })
      );
    });
  });

As I found in the JavaScript version of the tests, I have to introduce some minor time delays to allow the Polymer element to draw or update itself. But with carefully placed Timer runs, I have a second passing test:

PASS: [defaults] it has no toppings
PASS: [adding toppings] updates the pizza state accordingly

Those Timer delays are a little on the ugly side, but Page Objects again play very well with testing my Polymer element. There may be something to these Page Object things after all.


Day #10

‹prev | My Chain | next›

Testing Polymers with Page Objects (which I hate)

I hate everything about Page Objects. So surely I can be an impartial evaluator at applying them to testing Polymer, right?

OK, I don't hate everything about Page Objects. I loathe code reuse and indirection in tests. I am a testing fiend, but would rather let my code go untested than introduce indirection into my tests. I cannot count the number of hours that I have traced through other coder's helpers, behaves-likes, and attempts at DRYing up tests. I am decidedly of the opinion that test code should be un-DRY, procedural and entirely contained in the test or associated setup block. Simply put, debugging test helpers drives me absolutely nuts.

So why even consider Page Objects, which expose page- or component-level interaction methods? Mostly because I wound up introducing a helper for selecting items from a Polymer element's <select> last night:

  describe('adding a whole topping', function(){
    it('updates the pizza state accordingly', function(){
      var toppings = el.$.wholeToppings,
          select = toppings.$.ingredients,
          button = toppings.$.add;

      helpers.selectOption(select, 'green peppers');
      button.click();
      // The expectations are set later...
    });
  });

I need that helper because Polymer elements need actual native events to trigger bound variable updates:

var helpers = {
  selectOption: function(el, v) {
    var index = -1;
    for (var i=0; i<el.length; i++) {
      if (el.options[i].value == v) index = i;
    }
    el.selectedIndex = index;
    var event = document.createEvent('Event');
    event.initEvent('change', true, true);
    el.dispatchEvent(event);
  }
};

That code was sufficiently ugly enough to compel me, an test indirection hater, to introduce indirection into my test. And really, how much worse could Page Objects be?

I think it makes sense to leave the Polymer element creation to the test setup, so I define the Page Object constructor as accepting the element in question:

function XPizzaComponent(el) {
  this.el = el;
}

So the setup code is then responsible for adding the necessary <x-pizza> element (which builds custom pizzas) to the page for testing and then instantiating the XPizzaComponent object:

describe('', function(){
  var container, xPizza;

  beforeEach(function(){
    container = document.createElement("div");
    var el = document.createElement("x-pizza");
    container.appendChild(el);
    document.body.appendChild(container);

    xPizza = new XPizzaComponent(el);
    // Additional setup...
  });
  
  // Tests here...
});

Next, I need to define methods for my Page Object that return current information about the element and represent different ways that a user might interact with the element. Of late, I have been adding whole toppings to the pizza:

function XPizzaComponent(el) {
  this.el = el;
}
XPizzaComponent.prototype = {
  addWholeTopping: function(v) {
    var toppings = this.el.$.wholeToppings,
        select = toppings.$.ingredients,
        button = toppings.$.add;

    var index = -1;
    for (var i=0; i<select.length; i++) {
      if (select.options[i].value == v) index = i;
    }
    select.selectedIndex = index;
    var event = document.createEvent('Event');
    event.initEvent('change', true, true);
    select.dispatchEvent(event);

    button.click();

    return this;
  }
};

That is a less generalized version of last night's helpers.selectOption(). It may be less generalized, but it is far more meaningful in the context of my page / web component. Also YAGNI.

In addition to representing how a user interacts with my Polymer element, I also need to expose methods that can interrogate the element for current state. Of late, I have been working to test the current display state of the pizza, which is a JSON bound variable inside a <pre> tag. So I add a method for that as well to my Page Object:

function XPizzaComponent(el) {
  this.el = el;
}
XPizzaComponent.prototype = {
  addWholeTopping: function(v) { /* ... */ },
  currentPizzaStateDisplay: function() {
    if (this.el.$ === undefined) return '';
    return this.el.$.state.textContent;
  }
};

With that, and with my setup already instantiating this XPizzaComponent Page Object as xPizza, I can rewrite my tests as:

  describe('defaults', function(){
    it('has no toppings anywhere', function() {
      var no_toppings = JSON.stringify({
        firstHalfToppings: [],
        secondHalfToppings: [],
        wholeToppings: []
      });

      expect(xPizza.currentPizzaStateDisplay()).
        toEqual(no_toppings);
    });
  });

I grudgingly admit that this is fairly nice.

I am unsure how best to handle asynchronous code with this approach. The bound variable is not updated immediately after a user selects it from the drop-down. Rather, the browser needs to run through an event loop before Polymer updates bound variables. For now, I leave that in the hands of my tests rather than trying to force it into the Page Object:

  describe('adding a whole topping', function(){
    it('updates the pizza state accordingly', function(){
      xPizza.addWholeTopping('green peppers');

      // Wait a single event loop to allow Polymer to update the element…
      waits(0);
      runs(function(){
        var with_toppings = JSON.stringify({
          firstHalfToppings: [],
          secondHalfToppings: [],
          wholeToppings: ['green peppers']
        });

        expect(xPizza.currentPizzaStateDisplay()).
          toEqual(with_toppings);
      });
    });
  });

That aside, I again grudgingly admit that adding a topping and interrogating the display state are nicer with Page Objects. Darn it.

There may be something to this approach after all. But if anyone tries to clean up my addWholeToppings() method or factor any of this out into a separate library, there's gonna be trouble. Trouble I say!


Day #9

‹prev | My Chain | next›

Pretty Polymer Test Helpers

I actually thought testing my Polymer element last night would be pretty darn easy. As usual, I find the most trouble when I think a thing is going to be easy.

The Polymer element in question is a relatively simple pizza building element that I cleverly named <x-pizza>. The element itself is comprised of a <pre> element that holds the current state of the pizza being built and three child Polymer elements that are responsible for adding toppings to the pizza:



I ran into two problems with testing. First, it was hard to get the test to wait for the bound variable inside the <pre> to be updated. The bound variable was not updated on the next event loop as I would have preferred, but some time afterward. My solution was to wait for the <pre> to change, but I would have preferred a solution that did not wait on the same value being checked in the test. My second problem was that testing changes to <select> elements is a pain. I start with that pain tonight.

I wound up selecting the “green peppers” option from the <select> list with the following:

      select.selectedIndex = 3;
      var event = document.createEvent('Event');
      event.initEvent('change', true, true);
      select.dispatchEvent(event);

That's ugly-ish. Normally I would avoid jQuery, but I think it can help clean this up a little. So I add it to the list of Bower dev dependencies:

{
  "name": "mdv_example",
  // ...
  "dependencies": {
    "polymer": "Polymer/polymer"
  },
  "devDependencies": {
    "jquery": ">0"
  }
}

After a bower update, I modify my Karma configuration to pull in jQuery:

module.exports = function(config) {
  config.set({
    // ...
    files: [
      'test/PolymerSetup.js',
      'bower_components/jquery/dist/jquery.js',
      {pattern: 'elements/**', included: false, served: true},
      {pattern: 'bower_components/**', included: false, served: true},
      'test/**/*Spec.js'
    ],
    // ...
  });
};

Finally, I change the custom selectedIndex and manual event creation with:

      jQuery(select).val('green peppers').change();

That is a far more readable way of saying “select green peppers and send a change event.” So naturally, it does not work at all.

Well, it works a little. It does select “green peppers” from the <select> list, but it turns out that jQuery does not always generate native events. Yay.

So I drop jQuery as a development dependency and remove it from the list of Karma files. That is fine by me as I did not really want it in the first place. Also, I like my grapes on the sour side. Anyhow...

I replace my jQuery attempt with a custom written helper:

      helpers.selectOption(select, 'green peppers');

Which I define as:

var helpers = {
  selectOption: function(el, v) {
    var index = -1;
    for (var i=0; i<el.length; i++) {
      if (el.options[i].value == v) index = i;
    }
    el.selectedIndex = index;
    var event = document.createEvent('Event');
    event.initEvent('change', true, true);
    el.dispatchEvent(event);
  }
};

I can live with that.

The other problem that I faced from last night can be rephrased as wondering if I am testing the framework. I am striving for something close to an end-to-end acceptance test here, simulating user actions and checking the resulting changes to the DOM. The thing is, I am setting my expectation on a bound variable in the <x-pizza> template:

<link rel="import" href="../bower_components/polymer/polymer.html">
<polymer-element name="x-pizza">
  <template>
    <h2>Build Your Pizza</h2>
    <pre id="state">{{pizzaState}}</pre>
    <!-- ... -->
  </template>
  <script src="x_pizza.js"></script>
</polymer-element>

So by checking for the contents of the pre#state element, I am testing that Polymer is performing data binding properly. That is, I am testing the framework while running an acceptance test. So I could forgo setting expectations on the resulting DOM and instead set the expectation on the pizzaState property directly.

At least that is how I try to rationalize loosening my acceptance test. Fortunately, it turns out that I do not need to. Waiting for the variable bound <pre> to update is unnecessary in this case. I suspect that frustration made me sloppy last night, making me think I had to watch for a change. Instead, I merely need to wait a single browser event loop before checking my expectation. Thus the entire test of my <x-pizza> update can be written as:

    it('updates the pizza state accordingly', function(){
      var toppings = el.$.wholeToppings,
          select = toppings.$.ingredients,
          button = toppings.$.add;

      helpers.selectOption(select, 'green peppers');
      button.click();

      // Wait a single event loop to allow Polymer to update the element…
      waits(0);
      runs(function(){
        var with_toppings = JSON.stringify({
          firstHalfToppings: [],
          secondHalfToppings: [],
          wholeToppings: ['green peppers']
        });
        expect(el.$.state.textContent).toEqual(with_toppings);
      });
    });

Navigating the shadow DOM of the Polymer element and children at the outset of the test is a little awkward, but that is Polymer life. The rest of the test is quite nice, thanks to a simple, custom-written helper and a little Jasmine asynchronous testing love.

With that, I am much happier than I was at this time last night.


Day #8

‹prev | My Chain | next›

WaitsFor Polymer-Ready… FOREVER!

Tonight, I need to figure out how to test my <x-pizza> Polymer element. Really, I should have done this last night before refactoring, but sometimes the siren song of shiny new refactoring is too strong to resist. Thankfully there is always tomorrow. And by tomorrow, I mean today.

This version of the element is only a few select lists and and a JSON representation of the current pizza state:



Later on in Patterns in Polymer I give this an SVG makeover. For tonight, I only need test adding pizza toppings and checking expectations on the pizza state.

I start with the default value of the pizza state, which should be a JSON representation of a pizza with no toppings. In Jasmine-ese:

  describe('defaults', function(){
    it('has no toppings anywhere', function() {
      var el = document.querySelector('x-pizza');
      var pre = el.shadowRoot.querySelector('pre');

      var no_toppings = JSON.stringify({
        firstHalfToppings: [],
        secondHalfToppings: [],
        wholeToppings: []
      });

      expect(pre.textContent).toEqual(no_toppings);
    });
  });

I am fairly sure that this is the right test, but… it always fails due to the pizza state <pre> element being blank:

Chrome 33.0.1750 (Linux) <x-pizza> defaults has no toppings anywhere FAILED
        Expected '' to equal '{"firstHalfToppings":[],"secondHalfToppings":[],"wholeToppings":[]}'.
        ...

It seems that my test setup is not robust enough to handle a Polymer that requires child elements to load. I had been using a beforeEach() that waited for a single JavaScript event loop to allow the tested Polymer to ready itself for testing:

  beforeEach(function(){
    container = document.createElement("div");
    container.innerHTML = '';
    document.body.appendChild(container);
    waits(0); // One event loop for elements to register in Polymer
  });

If I futz with the waits() amount, I eventually find that 250ms is sufficient time to get this working. Sticking with a hard-coded value like that would be silly. It is clearly dependent on my machine and the current complexity of my <x-pizza> Polymer element. If I tried running the test on a slower machine or if I ever add more complexity, this waits() amount would no longer be sufficient and I would have a falsely failing test on my hands.

To get this to work, I first try to waitsFor() a polymer-ready event:

  beforeEach(function(){
    var _isPolymerReady = false;
    window.addEventListener('polymer-ready', function(e) {
      _isPolymerReady = true;
    });

    container = document.createElement("div");
    container.innerHTML = '';
    document.body.appendChild(container);

    waitsFor(
      function() {return _isPolymerReady;},
      "polymer-ready",
      5000
    );
  });

The trouble is that this event never fires:

timeout: timed out after 5000 msec waiting for polymer-ready

More accurately, it has already fired by the time my test setup is run. In reality, I do not want my test to wait for polymer-ready, I want my test to wait until this particular Polymer element is ready. And really, even that is not quite right. What I really need is to wait until the Polymer element is ready, the child elements are all ready, the internal pizza state has been updated, and the bound variable in the template has been updated.

Ugh. There is not exactly a lifecycle callback or event for that. In the end, I find that I have to craft a waitsFor() that blocks for two things: the Polymer $ property to be present and for the <pre> element to have content. The beforeEach() setup block for that is:

  beforeEach(function(){
    container = document.createElement("div");
    el = document.createElement("x-pizza");
    container.appendChild(el);
    document.body.appendChild(container);

    waitsFor(
      function() {
        return el.$ !== undefined &&
               el.$.state.textContent != '';
      },
      "element upgraded",
      5000
    );
  });

I am not particularly thrilled to be waiting for the value being tested, el.$.state.textContent, to be present.

It gets even worse when I try to select an option from the child element's select list:

  describe('adding a whole topping', function(){
    it('updates the pizza state accordingly', function(){
      var toppings = el.$.wholeToppings,
          select = toppings.$.ingredients,
          button = toppings.$.add;

      // Polymer won't see a select list change without a change event...
      select.selectedIndex = 3;
      var event = document.createEvent('Event');
      event.initEvent('change', true, true);
      select.dispatchEvent(event);

      button.click();

      var old = el.$.state.textContent;
      waitsFor(function(){return el.$.state.textContent != old;});
      runs(function(){
        var with_toppings = JSON.stringify({
          firstHalfToppings: [],
          secondHalfToppings: [],
          wholeToppings: ['green peppers']
        });

        expect(el.$.state.textContent).toEqual(with_toppings);
      });
    });
  });

Here, I change the value of one of the <select> menus, click the “Add Topping” button and expect that the chosen topping has been added to the current state of the pizza. In addition to again being required to waitsFor() the <pre> element to update, I find that I also have to manually trigger a change event in the <select> list. Without this event, Polymer does not update the bound variable associated with the <select> list, which prevents the topping from being added to the pizza.

All of this feels much harder than it ought to be. I am satisfied that I have crafted relatively robust tests, but they feel like more work that should be necessary. I will likely take another run at these two tests in another day or so. I like the acceptance test feel of querying the resulting <pre> tag, but some of this would go away if I queried the pizzaState property directly rather than setting the expectation on an element that is data bound to pizzaState. I may also pull in jQuery—at least in my tests—to make some of the event code a little easier. Grist for exploration tomorrow...


Day #7

‹prev | My Chain | next›

Refactoring in the Spirit of Polymer

One of the really solid pieces of advice that I have received on Patterns in Polymer is that the example in the Model Driven View chapter is a tad large.

The MDV example is a simple pizza maker:



The “model” part being a simple object literal comprised of different lists for the toppings:

Polymer('x-pizza', {
  ready: function() {
    this.model = {
      firstHalfToppings: [],
      secondHalfToppings: [],
      wholeToppings: []
    };
  },
  // ...
});

I favor this approach because the model, which is the central piece of the MDV chapter, it relatively small and easily understood. That said, the code that backs these lists is repetitive and long. In other words, the backing class strays from the “Polymer way.” I hate to lose the current, easily understood model, but I also hate to stray from the spirit of Polymer in any of my examples.

I may very well have to introduce an entirely different example. Before I go to that extreme, I try one of the suggestions of moving the model into a new <x-pizza-toppings> Polymer element. This initially means that the <x-pizza> template gets much simpler. It goes from:

<polymer-element name="x-pizza">
  <template>
    <p>
      <select class="form-control" value="{{currentFirstHalf}}">
        <option>Choose an ingredient...</option>
        <option value="{{ingredient}}" template repeat="{{ingredient in ingredients}}">
          {{ingredient}}
        </option>
      </select>
      <button on-click="{{addFirstHalf}}" type="button" class="btn btn-default">
        Add First Half Topping
      </button>
    </p>
    <!-- Nearly identical 2nd half and whole toppings template code... -->
  </template>
  <script src="x_pizza.js"></script>
</polymer-element>

To the much more readable:

<link rel="import" href="x-pizza-toppings.html">
<polymer-element name="x-pizza">
  <template>
    <h2>Build Your Pizza</h2>
    <pre>{{pizzaState}}</pre>
    <x-pizza-toppings id="firstHalfToppings"
                      name="First Half Toppings"
                      ingredients="{{ingredients}}"></x-pizza-toppings>
    <x-pizza-toppings id="secondHalfToppings"
                      name="Second Half Toppings"
                      ingredients="{{ingredients}}"></x-pizza-toppings>
    <x-pizza-toppings id="wholeToppings"
                      name="Whole Toppings"
                      ingredients="{{ingredients}}"></x-pizza-toppings>
  </template>
  <script src="x_pizza.js"></script>
</polymer-element>

The repetitive HTML goes into the template for <x-pizza-toppings> with almost no changes. The backing class of <x-pizza-toppings> then gets the model:

Polymer('x-pizza-toppings', {
  ingredients: [],
  ready: function() {
    this.model = [];
  },
  current: '',
  add: function() {
    this.model.push(this.current);
  }
});

That really is much cleaner. The “model” is now an array, which is a little weird, but I can live with that. The code is much cleaner and much DRYer, so this definitely feels better.

That said, I am more than a little concerned at the number of concepts that I would be introducing with this example. In addition to discussing MDV, I would also have to introduce child Polymer elements. I would even have to mention the data binding of the master ingredients list in <x-pizza> for sharing the list with the child elements. This is not too horrible, especially for a book that is aimed at beyond the introduction. Still, the fewer the concepts, the better.

Speaking of concepts, I am still not quite done here. In the old version, I updated a string representation of the entire pizza whenever a topping change was made. To do that in this version, I need the child <x-pizza-toppings> elements to communicate up to the parent <x-pizza> element whenever a change occurs. That means observing the <x-pizza-toppings> model for changes so that it can fire a custom event:

Polymer('x-pizza-toppings', {
  observe: {
    'model': 'fireChange'
  },
  // ...
  fireChange: function() {
    this.fire('topping-change');
  }
});

The <x-pizza> can then listen for these topping change events, updating the “pizza state” accordingly:

Polymer('x-pizza', {
  // ...
  ready: function() {
    this.addEventListener('topping-change', function(event){
      this.updatePizzaState();
    });
  },
  updatePizzaState: function() {
    var pizzaState = {
      firstHalfToppings: this.$.firstHalfToppings.model,
      secondHalfToppings: this.$.secondHalfToppings.model,
      wholeToppings: this.$.wholeToppings.model
    };
    this.pizzaState = JSON.stringify(pizzaState);
  }
});

With that, I have a fully functional <x-pizza> Polymer element:



I have much less code accomplishing this and it all feels much more approachable. Still...

I will need to think about this approach before pulling it into the book. The ultimate solution may be as simple as only discussing <x-pizza-toppings> and leaving the inclusion in <x-pizza> until a later chapter. That would require some chapter reorganization, but it may be worth it.

Regardless, something needs to change because, thanks to some very helpful feedback, I am much happier with this solution than my previous approach. Improved solutions are always worth the effort, so I have some editing ahead of me!


Day #6

‹prev | My Chain | next›