Gladius, Long and Windy

‹prev | My Chain | next›

I remain unsure what to work on for my next book. There was much interest in Javascript gaming. I do not know it well enough to gauge how mature it is, but I know how to find out...

I am going to start with Gladius, the 3d gaming engine from Mozilla. First up, I clone it locally:

➜  repos  git clone https://github.com/alankligman/gladius-core.git
Cloning into 'gladius-core'...

That takes a little while, but once I have the repository, it seems that I need to make setup:

➜  gladius-core git:(develop) make setup
make: *** No rule to make target `setup'.  Stop.

Or not.

Hrm... there's not even a Makefile in there (and what is a Jakefile?):

➜  gladius-core git:(develop) ls -1
Jakefile
lib
LICENSE
node_modules
package.json
README.md
src
tests
tools

The Jakefile seems to be a node thing, but there is no mention of setup. Gladius core bundles several NPM modules, but jake is not one of them:

➜  gladius-core git:(develop) npm ls
/home/chris/repos/gladius-core
├── amdefine@0.0.2 
├─┬ jshint@0.6.3 
│ ├── argsparser@0.0.6 
│ └─┬ minimatch@0.0.5 
│   └── lru-cache@1.0.6 
├─┬ qunit@0.5.1 
│ ├── argsparser@0.0.6 
│ ├─┬ bunker@0.1.1 
│ │ └─┬ burrito@0.2.11 
│ │   ├── traverse@0.5.2 
│ │   └── uglify-js@1.0.7 
│ ├─┬ cli-table@0.0.1 
│ │ └── colors@0.3.0 
│ ├── tracejs@0.1.4 
│ └── underscore@1.3.3 
├── requirejs@1.0.8 
└── uglify-js@1.2.6 

Let's see if installing it globally (so that it can run as an executable) will work:

➜  gladius-core git:(develop) npm install -g jake
npm http GET https://registry.npmjs.org/jake
npm http 200 https://registry.npmjs.org/jake
npm http GET https://registry.npmjs.org/jake/-/jake-0.2.33.tgz
npm http 200 https://registry.npmjs.org/jake/-/jake-0.2.33.tgz
/home/chris/local/node-v0.6.15/bin/jake -> /home/chris/local/node-v0.6.15/lib/node_modules/jake/bin/cli.js
jake@0.2.33 /home/chris/local/node-v0.6.15/lib/node_modules/jake

And...

➜  gladius-core git:(develop) jake setup
jake aborted.
Error: Cannot find module 'jake'
    at Function._resolveFilename (module.js:332:11)
(See full trace by running task with --trace)

After running with --trace, I find that the error is coming from the Jakefile. Despite not being bundled with the rest of the NPM packages, it seems that I really do need it:

➜  gladius-core git:(develop) npm install jake
npm http GET https://registry.npmjs.org/jake
npm http 304 https://registry.npmjs.org/jake
npm WARN prefer global jake@0.2.33 should be installed with -g
jake@0.2.33 ./node_modules/jake

Yah, no kidding, NPM. I tried to install it globally and that didn't work.

After all that, it turns out that there is no setup task in the Jakefile or in the included files:

➜  gladius-core git:(develop) jake setup
jake aborted.
Error: Unknown task "setup"
    at [object Object].nextPrereq (/home/chris/local/node-v0.6.15/lib/node_modules/jake/lib/task/task.js:154:19)
(See full trace by running task with --trace)

So what happens if I just run the default task?

➜  gladius-core git:(develop) jake
jake aborted.
TypeError: Object [object Object] has no method 'exec'
    at [object Object].action (/home/chris/repos/gladius-core/tools/jake-tasks/default.js:15:8)
(See full trace by running task with --trace)

Sigh. It seems that the "jake tasks" included in the Jakefile expect a locally installed jake. So I explicitly run that locally installed jake and am able to build (but still not setup):

➜  gladius-core git:(develop) ./node_modules/.bin/jake build

Tracing dependencies for: ../tools/almond

/home/chris/repos/gladius-core/dist/gladius-core.js
----------------
/home/chris/repos/gladius-core/lib/../tools/almond.js
/home/chris/repos/gladius-core/lib/_math.js
...
/home/chris/repos/gladius-core/lib/../src/core/engine.js
/home/chris/repos/gladius-core/lib/../src/core/gladius-core.js

That seems to have created a gladius-code.js in dist:

➜  gladius-core git:(develop) find . -mmin -10         
.
./.git
./.git/index
./dist
./dist/gladius-core.js
./dist/gladius-core.min.js

So now it's time to use this. Since I already seem to be in node-land, I install express.js globally and use that to generate an express.js site for myself:

cd $HOME/repos
mkdir gladius-my
cd gladius-my
git init .
npm install -g express
express .
npm install

I comment out the default, "index" route in the generated apps.js:

// ...
// Routes

// app.get('/', routes.index);
// ...

Instead, I copy the contents of the example page into public/index.html. I do update the <script> tag to point to gladius-core.js in express.js' /javascripts/ path:

...
<head>
  <script src='/javascripts/gladius-core.js'>
  </script>
...

Lastly, I copy the generated gladius-core.js files into publics/javascripts:

cp ../gladius-core/dist/gladius-core.* public/javascripts

And...

That does not work. It seems that gladius has switched to requirejs for library loading. Bah! I like requirejs, but that marks one too many obstacles to overcome in one night. I will pick back up with this tomorrow.



Day #403

Dartium Snapshotting in Action (sort of)

‹prev | My Chain | next›

Last night I specially compiled a version of Dartium that can spapshot VM images of Dart applications. I am unsure what that actually means in practice, so tonight I am going to try to find out.

I start with the Speed Trace extension for Chrome / Chromium / Dartium. I used it to watch a fresh load of my Hipster MVC-based application: Dart Comics. Looking at browser events in Speed Tracer, I see:

I do not see anything obvious in that graph. The script evaluation is 50ms, but I am unsure if that includes Dart code or just Javascript (which is still required to kickstart the Dart VM).

Taking a look at the network events, I see the many Dart classes comprising my application and Hipster MVC being loaded:

So what happens if I reload the application? Does the script evaluation time go down because the VM is in cache?

In fact there is no change. If snapshotting is really in effect and Speed Tracer is including Dart evaluation in its calculation, then I ought to see a decrease. Regardless, the Speed Tracer extension is not going to be of any use to me since I can see the network information just as easily in Dartium's Developer Tools.

So what do I see when I do a reload?

Oooh! That is interesting. I see several 304 / Not Modifieds as the page, CSS, JS (starts the Dart engine), and main Dart script are reloaded. But the next resource loaded is the MVC framework loading my comic book collection—not the remainder of my application code being reloaded. Is that snapshotting in effect or does Dartium just always do that?

I still have the compiled Dartium available, so I load the application, then reload and see:

Ah, finally, snapshotting does have an effect. When enabled, Dartium checks the main.dart entry point. If it is unmodified, then Dartium does not even bother checking the remainder of the Dart code.

So does this caching linger between restarts? To answer, I close Dartium, start it back up and load Dart Comics:

Bummer. Dartium still loads all of the application resources even though main.dart is 304 / Not Modified.

Per a discussion on the mailing list, the snapshop enabled version of Dartium ought to honor a misnamed --enable-preparsed-js-caching command line switch for writing VMs to disk. Perhaps this will have an effect?

I start Dartium with said command line option:

➜  ~  ./repos/dartium/src/out/Release/chrome --enable-preparsed-js-caching

Then I load the application and close the browser. Hopefully this will write the VM to disk. I start Dartium, again with the --enable-preparsed-js-caching command line switch and load Dart Comics to see:

Aw well, it seems that snapshotting does not survive restarts. At least not yet. This is, after all, a disabled feature that needs special compilation options to enable. Presumably this will get more sophisticated over time.

What I would really like to see is actual VM files that can be loaded over HTTP for faster start up. Or better yet, pushed directly into browser cache with SPDY server push. That could be amazing.

Day #402

Getting Started with Dart Snapshots

‹prev | My Chain | next›

Among the myriad of promises offered by Dart is snapshotting of VM images. My understanding is that VM snapshots will not be like Squeak images—that is Dart snapshots will not contain state information. Rather they will hold compile-time snapshots of the classes employed. These snapshots can be cached so that the next time that the browser request the Dart application, compilation can be by-passed for even faster startup. At least that's my understanding. I could be wrong.

A while back, Anton Muhin was kind enough to give me some pointers for getting it working. My compile machine died, but it has since returned from the dead.

I update src/third_party/WebKit/Source/WebCore/bindings/dart/DartApplicationLoader.cpp to enable snapshotting:

// ...
static bool isSnapshottingEnabled()
{
    return true;
}
// ...

Look at me, I'm a C++ programmer! Well, not quite, but I can compile it:

➜  src  ./dartium_tools/build.py --mode=Release                         
Running:  make -j8 BUILDTYPE=Release chrome DumpRenderTree test_shell dart
  CXX(target) out/Release/obj.target/webcore_remaining/third_party/WebKit/Source/WebCore/bindings/dart/DartApplicationLoader.o
make: Nothing to be done for `dart'.
  AR(target) out/Release/obj.target/third_party/WebKit/Source/WebCore/WebCore.gyp/libwebcore_remaining.a
  TOUCH out/Release/obj.target/third_party/WebKit/Source/WebCore/WebCore.gyp/webcore.stamp
  LINK(target) out/Release/chrome
  LINK(target) out/Release/DumpRenderTree
  LINK(target) out/Release/test_shell
  LINK(target) out/Release/chrome: Finished
  LINK(target) out/Release/DumpRenderTree: Finished
  LINK(target) out/Release/test_shell: Finished

When I run this again my Dart Comics application, however, I am greeted by a sad browser:

And the following STDERR output from Dartium:

dart/runtime/vm/raw_object_snapshot.cc:1094: error: unimplemented code

And indeed, line 1094 of raw_object_snapshot.cc is undefined:

void RawLanguageError::WriteTo(SnapshotWriter* writer,
                               intptr_t object_id,
                               Snapshot::Kind kind) {
  UNIMPLEMENTED();
}

Hrm... But why is that a RawLanguageError::WriteTo? The "Error" portion of that function is not encouraging. Grepping through the code, it seems like this may be coming from snapshot.cc in the same directory:

...
  switch (kind) {
#define SNAPSHOT_WRITE(clazz)                                                  \
    case clazz::kInstanceKind: {                                               \
      Raw##clazz* raw_obj = reinterpret_cast(raw);                \
      raw_obj->WriteTo(this, object_id, kind_);                                \
      return;                                                                  \
    }                                                                          \

    CLASS_LIST_NO_OBJECT(SNAPSHOT_WRITE)
#undef SNAPSHOT_WRITE
...

Ugh. Macros. Why did it have to be CPP macros?

Kidding aside, I did not know that you could pass a macro to another macro to be invoked. But that is what CLASS_LIST_NO_OBJECT does—invokes SNAPSHOT_WRITE for every class that it defines, including LanguageError:

// runtime/vm/raw_object.h
// ...
// Macrobatics to define the Object hierarchy of VM implementation classes.
#define CLASS_LIST_NO_OBJECT(V)                                                \
  V(Class)                                                                     \
  V(UnresolvedClass)                                                           \
  V(AbstractType)                                                              \
    V(Type)                                                                    \
    V(TypeParameter)                                                           \
  V(AbstractTypeArguments)                                                     \
    V(TypeArguments)                                                           \
    V(InstantiatedTypeArguments)                                               \
  V(Function)                                                                  \
...
  V(Error)                                                                     \
    V(ApiError)                                                                \
    V(LanguageError)                                                           \
    V(UnhandledException)                                                      \
    V(UnwindError)                                                             \
...

And still, that does not help. Why is this a RawLanguageError WriteTo?

Unfortunately, I am out of ideas, so I resort to removing my Dartium profile:

rm -rf .config/chromium

Which has no effect. I had not really expected it to since my Chromium profile ought to be next to empty.

I am out of my depth in C++ and, more specifically C++ / browser debugging so I will see if a kind soul on the mailing list might take pity on a poor interpreted language developer.

UPDATE: I figured it out. Eventually, I noodled out that "LanguageError" might mean that I had a problem with my Dart code. So I enabled type checking and assertions by restarting my (Linux) Dartium as:

DART_FLAGS='--enable_type_checks --enable_asserts' dartium

Indeed there were some errors (mostly cruft from one too many bleeding edge experiments). After resolving those issues, I was able to access my Dart Comics application. I still need to experiment with the actual results of snapshotting, but I will leave that for another night.

Day #401

Dart Pub for Local Development

‹prev | My Chain | next›

I got a nice taste of the pub packager for Dart (it's so new, it has a commit, not a landing page). Like just about everything else in Dart, pub is starting small, with just enough to get an idea how it might work, but not so much that it is at all encumbered by previous implementation.

Tonight, I'd like to see how far I can push the current implementation. Specifically, I would like to see if I can make it manually emulate npm's very excellent npm link. The "link" in that command instructs npm to link a local copy of a package instead of installing it from the npm registry. It is more than installing it from a local copy—it creates a symbolic link to the local package instead of installing it. The invaluable benefit of this approach is that any changes made to the linked package can be immediately committed without the intermediate step of copying them into a separate repository.

I do this constantly as I am developing Hipster MVC. Try something new in Dart Comics / tweak the code in Hipster MVC. Struggle with an implementation aspect in Dart Comics / update the documentation in Hipster MVC.

But until now, this has been a pain. I either need to update the code on GitHub to see the change locally:

...
#import('https://raw.github.com/eee-c/hipster-mvc/master/HipsterSync.dart');
...

Or I have to manually sym-link the Hipster MVC libraries for local development only to remove them (and the updated #import() statements) before I commit to the Dart Comics repository.

So I get started by replacing the remaining references to GitHub-hosted code in Dart Comics with the pub package equivalent:

...
#import('package:hipster-mvc/HipsterSync.dart');
...

Next, in the scripts directory of Dart Comics, I replace the pub installed Hipster MVC with a symbolic link to my local copy of the repository:

➜  scripts git:(pub-packages) cd packages
➜  packages git:(pub-packages) mv hipster-mvc hipster-mvc.orig
➜  packages git:(pub-packages) ln -s ../../../../hipster-mvc
➜  packages git:(pub-packages) ls -l
total 4
lrwxrwxrwx 1 chris chris   23 May 28 22:02 hipster-mvc -> ../../../../hipster-mvc
drwxr-xr-x 3 chris chris 4096 May 27 23:06 hipster-mvc.orig

And, indeed, it all still works. I can view, delete, and add comic books to my collection just as before:

Best of all, if I make a change to my local copy of Hipster MVC, such as breaking forEach():

class HipsterCollection implements Collection {
  // ...
  void forEach(fn) {
    //models.forEach(fn);
  }
  // ...
}

Then the change is immediately visible in my Dart Comics application:

And if I fix the change in the local Hipster MVC repository:

class HipsterCollection implements Collection {
  // ...
  void forEach(fn) {
    models.forEach(fn);
  }
  // ...
}

Then Dart Comics is fixed:

That is pretty freaking amazing. If there is anything even close in client-side Javascript land, I am unaware. But even is a pre-release / proof-of-concept, the pub packager already makes life oh-so-sweet for the Dart developer.

Am I overstating? This seems huge.


Day #400

Beginning Pub Packaging in Dart

‹prev | My Chain | next›

I have been pleasantly surprised at how well my old Dart code has held up. I am also glad that my grasp of the language has not deteriorated during my SPDY break. I am left in the somewhat unexpected state of not needing to do as much as I had planned to keep things fresh as I refresh some of the chapters in Dart for Hipsters. Still, I want to keep focused on Dart for a little while longer, so I will take some time tonight to look at something that definitely popped up while I was on sabbatical: the pub packaging tool.

I am grateful to Allan MacDonald for pointing me to a mailing list discussion on this very topic in which Bob Nystrom describes how to use pub in the dart sdk, which I conveniently have.

So I return to my Dart-powered, client-side comic book application. It turns out that I am in dire need of some help here working with the Hipster MVC library. As I make changes to the application itself, I invariably need to make a change to Hipster MVC. And I wind up in a state something like:

➜  dart-comics git:(modal-dialog) ✗ git status
# On branch modal-dialog
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
# new file:   public/scripts/HipsterCollection.dart
# new file:   public/scripts/HipsterHistory.dart
# new file:   public/scripts/HipsterModel.dart
# new file:   public/scripts/HipsterRouter.dart
# new file:   public/scripts/HipsterSync.dart
# new file:   public/scripts/HipsterView.dart
#
# Changes not staged for commit:
#   (use "git add ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
# modified:   public/scripts/Collections.Comics.dart
# modified:   public/scripts/HipsterCollection.dart
# modified:   public/scripts/HipsterHistory.dart
# modified:   public/scripts/HipsterModel.dart
# modified:   public/scripts/HipsterRouter.dart
# modified:   public/scripts/HipsterSync.dart
# modified:   public/scripts/HipsterView.dart
# modified:   public/scripts/ModalDialog.dart
# modified:   public/scripts/Models.ComicBook.dart
# modified:   public/scripts/Views.AddComic.dart
# modified:   public/scripts/Views.AddComicForm.dart
# modified:   public/scripts/Views.Comics.dart
# modified:   public/scripts/main.dart
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
# public/scripts/ComicBook.dart
# public/scripts/ComicBook.tmpl

A lot of that is actual changes to dart comics, but there is also much noise added by removing local copies and replacing them with symbolic links to my local Hipster MVC repository:

➜  dart-comics git:(modal-dialog) ✗ ls -l public/scripts/Hipster*
lrwxrwxrwx 1 chris chris 43 May 24 22:26 public/scripts/HipsterCollection.dart -> ../../../hipster-mvc/HipsterCollection.dart
lrwxrwxrwx 1 chris chris 40 May 24 22:26 public/scripts/HipsterHistory.dart -> ../../../hipster-mvc/HipsterHistory.dart
lrwxrwxrwx 1 chris chris 38 May 24 22:26 public/scripts/HipsterModel.dart -> ../../../hipster-mvc/HipsterModel.dart
lrwxrwxrwx 1 chris chris 39 May 24 22:26 public/scripts/HipsterRouter.dart -> ../../../hipster-mvc/HipsterRouter.dart
lrwxrwxrwx 1 chris chris 37 May 24 22:26 public/scripts/HipsterSync.dart -> ../../../hipster-mvc/HipsterSync.dart
lrwxrwxrwx 1 chris chris 37 May 24 22:26 public/scripts/HipsterView.dart -> ../../../hipster-mvc/HipsterView.dart

Anyhow, hopefully pub can help somewhat—even in its infancy.

I remove those local copies of Hipster MVC and am left with just the local changes, of which there are many. With the clutter out of the way, I have a look at those changes to see that many of them are replacing references to the raw GitHub resources with local references:

diff --git a/public/scripts/main.dart b/public/scripts/main.dart
index da0067b..21c0bd8 100644
--- a/public/scripts/main.dart
+++ b/public/scripts/main.dart
@@ -5,7 +5,7 @@
 #import('dart:html');
 #import('dart:json');
 
-#import('https://raw.github.com/eee-c/hipster-mvc/master/HipsterSync.dart');
+#import('HipsterSync.dart');
 
 main() {
   // HipsterSync.sync = localSync;

Hrm... I don't think that pub is going to help much when I need to edit local copies instead of use the remote resources—at least not yet. Hopefully at some point, pub will get something like npm's link, which allows developers to override package management installs with local copies. I won't worry about that too much since pub is little more than preview release.

One of the first things that my dart comics application does is load in my comics collection from the backend:

main() {
  var my_comics_collection = new Collections.Comics()
    , comics_view = new Views.Comics(
        el:'#comics-list',
        collection: my_comics_collection
      );

  my_comics_collection.fetch();
  // ...
}

I think that Collections.Comics may be ripe for pub-ification:

#library('Collection class to describe my comic book collection');

#import('https://raw.github.com/eee-c/hipster-mvc/master/HipsterCollection.dart');

#import('Models.ComicBook.dart');

class Comics extends HipsterCollection {
  get url() => '/comics';
  modelMaker(_) => new ComicBook();
}

Specifically, that #import() from raw.github.com would be much cleaner as a package line:

#library('Collection class to describe my comic book collection');

#import('package:hipster-mvc/HipsterCollection.dart');

#import('Models.ComicBook.dart');

class Comics extends HipsterCollection {
  get url() => '/comics';
  modelMaker(_) => new ComicBook();
}

If I try to load the app with the package line, I get the following in the Dart console:

Failed to load resource: the server responded with a status of 404 (Not Found)
   http://localhost:3000/scripts/packages/hipster-mvc/HipsterCollection.dart

Nice! It seems that my Dartium is, in fact, pub-aware since the filed URL was in scripts/packages.

In my scripts directory, I create a pubspec file that describes dart-comics' dependencies:

dependencies:
  hipster-mvc:
    git: git://github.com/eee-c/hipster-mvc.git

With that, I am ready to pub-install my Hipster MVC dependency:

➜  scripts git:(modal-dialog) ✗ pub install
Cloning into '/home/chris/repos/dart-comics/public/scripts/packages/hipster-mvc'...
remote: Counting objects: 859, done.
remote: Compressing objects: 100% (174/174), done.
remote: Total 859 (delta 755), reused 780 (delta 676)
Receiving objects: 100% (859/859), 434.36 KiB | 127 KiB/s, done.
Resolving deltas: 100% (755/755), done.
Dependencies installed!

And indeed, I do have a packages directory:

➜  scripts git:(pub-packages) ✗ gst
# On branch pub-packages
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#       modified:   Collections.Comics.dart
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       packages/
#       pubspec
no changes added to commit (use "git add" and/or "git commit -a")

And Hipster MVC is in there:

➜  scripts git:(pub-packages) ✗ find packages | grep -v \\.git
packages
packages/hipster-mvc
packages/hipster-mvc/HipsterHistory.dart
packages/hipster-mvc/HipsterModel.dart
packages/hipster-mvc/HipsterSync.dart
packages/hipster-mvc/HipsterRouter.dart
packages/hipster-mvc/README.asciidoc
packages/hipster-mvc/HipsterView.dart
packages/hipster-mvc/HipsterCollection.dart
packages/hipster-mvc/main.dart

Most importantly, my dart comics application is back in business:

Last up tonight, I add the packages to the list of files ignored by git:

➜  scripts git:(pub-packages) ✗ echo packages >> .gitignore

There is no reason that I would ever want that directory checked into my git repository—especially since the packages directory is made up of git repositories.

That was a pretty nice first experience with pub packaging. I think I will continue playing with it tomorrow.


Day #399

Unlazily Static

‹prev | My Chain | next›

I ended up with an uglyish solution for mapping arrow keycodes to movement directions in some Dart code:

HashMap<int, String> _dir;
String keyDirection(int dir) {
  if (_dir) return _dir[dir];

  _dir = {};
  _dir[37] = 'left';
  _dir[38] = 'up';
  _dir[39] = 'right';
  _dir[40] = 'down';

  return _dir[dir];
}

The private variable _dir was bad form on my part—I was trying to avoid a perceived initialization penalty before I was actually affected by it. So I should have done something like:

String keyDirection(int dir) {
  var dir = {};
  dir[37] = 'left';
  dir[38] = 'up';
  dir[39] = 'right';
  dir[40] = 'down';

  return dir[dir];
}

In all honestly, even that is not terribly satisfactory. I would prefer to declare dir as:

var dir = {
  37: 'left',
  38: 'up',
  39: 'right',
  40: 'down'
};

Dart, however, does not allow object literal syntax to use integers as keys, so I am stuck with the individual assignments of the directions. And that continues to bother my sensibilities.

An astute reader suggest that I forgo the keyDirect() method entirely. Embracing the object-literal-with-string-key syntax, my key down handler can be written as:

  document.
    on.
    keyDown.
    add((event) {
      String direction = {
        '37': 'left',
        '38': 'up',
        '39': 'right',
        '40': 'down'
      }[event.keyCode.toString()];

      if (direction == null) return;

      event.preventDefault();
      me.move(direction);
      draw(me, context);
    });

That works and is better than what I had. Still, the actual lookup is obscured too much for my tastes. I would rather it be done via:

  document.
    on.
    keyDown.
    add((event) {
      String direction = _dirMap[event.keyCode.toString()];
      if (direction == null) return;

      event.preventDefault();
      me.move(direction);
      draw(me, context);
    });

I cannot simply declare _dirMap as a global variable:

// Fail: not a compile time constant
HashMap _dirMap = {
  '37': 'left',
  '38': 'up',
  '39': 'right',
  '40': 'down'
};

attachMover(me, context) {
  // ...
}

To make that work, I can make the left hand side of the _dirMap assignment a compile time constant with the simple addition of the const keyword:

HashMap const _dirMap = {
  '37': 'left',
  '38': 'up',
  '39': 'right',
  '40': 'down'
};

attachMover(me, context) {
  // ...
}

This brings me to another facet of the update Dart language spec: lazy evaluation of static variables.

In order to try this, I convert my entire game board into a Room class that includes a static variable version of _dirMap:

class Room {
  Player player;
  CanvasRenderingContext context;

  static HashMap _dirMap =  {
    '37': 'left',
    '38': 'up',
    '39': 'right',
    '40': 'down'
  };

  Room.play(this.player, this.context) {
    this.draw();
    this.attachMover();
  }

  draw() { /* ... */ }

  attachMover() {
    document.
      on.
      keyDown.
      add((event) {
        String direction = _dirMap[event.keyCode.toString()];
        if (direction == null) return;

        event.preventDefault();
        player.move(direction);
        draw();
      });
  }
}

My board is still working on load:

So maybe this works?

Uh, no. When I make my first move, I am greeted with the follow exception:

Internal error: 'file:///home/chris/repos/dart-book/book/includes/animation/main.dart': Error: line 37 pos 28: initializer must be a compile time constant
  static HashMap _dirMap = {
                           ^

So it seems that despite the spec, static variables are still unlazily evaluated. I add const to the Room class:

class Room {
  // ...
  static HashMap _dirMap = const {
    '37': 'left',
    '38': 'up',
    '39': 'right',
    '40': 'down'
  };
  // ...
}

And call it a night.

I am not too fussed by this. I would have been cool if this had made it into Dartium, but this means less work is needed to make a proper first release of Dart for Hipsters.


Day #398

Still Switch

‹prev | My Chain | next›

Ah, Dart. How I missed thee.

It really is a shame that there a no paying jobs in Dart just yet—it's just that fun (and I'm not just saying that because I'm writing the book). Thankfully, I have at least a few days to play with it again and the built-in excuse that I need to sanity check facts for the book.

Last night I was able to get the Hipster MVC library updated to work with the latest SDK in relatively short order. With nothing breaking, I can now move on through the language specification to see what else may have changed.

I note that the switch statement changed. Way back in March, I decided that the old implementation was not for me. Mayhaps it is now.

The code that I tried to switch mapped keyboard events to directions in a simple Dart animation:

attachMover(me, context) {
  // Move on key down
  document.
    on.
    keyDown.
    add((event) {
      String direction;

      if (event.keyCode == 37) direction = 'left';
      if (event.keyCode == 38) direction = 'up';
      if (event.keyCode == 39) direction = 'right';
      if (event.keyCode == 40) direction = 'down';

      if (direction != null) {
        event.preventDefault();
        me.move(direction);
        draw(me, context);
      }
    });

  // ...
}

Back in March, the best I could do was:

   switch(event.keyCode) {
        case 37: direction = 'left'; break;
        case 38: direction = 'up'; break;
        case 39: direction = 'right'; break;
        case 40: direction = 'down'; break;
      };

The combination of the direction assignment and the break for each case was enough to turn me off.

Ideally, I would like the switch statement to return the cased statement:

      direction = switch(event.keyCode) {
        case 37: 'left';
        case 38: 'up';
        case 39: 'right';
        case 40: 'down';
      };

Failing that, I might settle for a simple assignment statement:

      switch(event.keyCode) {
        case 37: direction = 'left';
        case 38: direction = 'up';
        case 39: direction = 'right';
        case 40: direction = 'down';
      };

Sadly, not much has changed. In particular, if I remove the break statements, I am greeted with a very detailed stacktrace:

Exception: 'file:///home/chris/repos/dart-book/book/includes/animation/main.dart': Switch case fall-through at line 70.
Stack Trace:  0. Function: 'FallThroughError._throwNew@127eafe4' url: 'bootstrap' line:464 col:178
 1. Function: '::function' url: 'file:///home/chris/repos/dart-book/book/includes/animation/main.dart' line:70 col:9
 2. Function: '_EventListenerListImpl@33cc944a.function' url: '/mnt/data/b/build/slave/dartium-lucid64-inc/build/src/out/Release/obj/gen/webkit/bindings/dart/html/dartium/EventTarget.dart' line:16 col:72

Well, at least it fairly explicit that the break statements are required to prevent said fall through.

Reading the spec a bit closer, it seems that the change was that the case value needs to be of the same type. That has no effect on my code since all of my cases are integers (37, 38, 39, 40).

Bah! I'll not be spending any more time on switch. But I do take some time to make the resolution of direction a little cleaner:

attachMover(me, context) {
  // Move on key down
  document.
    on.
    keyDown.
    add((event) {
      String direction = keyDirection(event.keyCode);
      if (direction == null) return;

      event.preventDefault();
      me.move(direction);
      draw(me, context);
    });
}

// Sadly hash literals are still not compile-time
// constants
HashMap<int, String> _dir;
String keyDirection(int dir) {
  if (_dir) return _dir[dir];

  _dir = {};
  _dir[37] = 'left';
  _dir[38] = 'up';
  _dir[39] = 'right';
  _dir[40] = 'down';

  return _dir[dir];
}

And I still have my little canvas experiment working and the red square "avatar" responding to arrow key input:

Tomorrow, I will continue to root through the the updated language spec to see if there is anything that I might have missed while I was away.


Day #397

Dart Revisited

‹prev | My Chain | next›

Dart for Hipsters is making steady progress at the new publisher. It's crunch time and I need to double-check that everything still works and if there is anything new that warrants inclusion. Now that I am confident in node-spdy's spdy/3 implementation, I have some time to revisit Dart.

My copy of Dartium and the sdk are quite old so I download both. I was somewhat concerned that Dartium might not start up on my 64 bit Ubuntu 12.04 system, but I still seem to have all of the necessary system packages installed:

Since I have a working Dartium, I try pointing it at my simple dart-comics application (manages my comic book collection), which is built on top of Hipster MVC. I see the main page of the app, but nothing seems to be working. In the Dart console, I see:

Internal error: 'http://localhost:3000/scripts/HipsterCollection.dart': Error: line 100 pos 26: class 'CollectionEventList' overrides function 'add' of interface 'EventListenerList' with incompatible parameters
  CollectionEventList add(fn) {
                         ^

Hrm... The add() method in CollectionEventList accepts a single parameter—a callback function to be invoked when the event fires:

class CollectionEventList implements EventListenerList {
  // ...
  CollectionEventList add(fn) {
    listeners.add(fn);
    return this;
  }
  // ...
}

The add() method in EventListenerList requires just the callback, but it also supports an optional boolean describing whether the listener applies in capture or bubble event phases:

EventListenerList add(EventListener handler, [bool useCapture]);

Do I need type information on the callback or the optional boolean? I opt to try the latter:

class CollectionEventList implements EventListenerList {
  // ...
  CollectionEventList add(fn, [bool useCapture]) {
    listeners.add(fn);
    return this;
  }
  // ...
}

That actually does the trick as I now see my beautiful app once again:

Interestingly, the name of the optional parameter seems to be important. If I drop the "e", I still get an error:

Internal error: 'http://localhost:3000/scripts/HipsterCollection.dart': Error: line 100 pos 26: class 'CollectionEventList' overrides function 'add' of interface 'EventListenerList' with incompatible parameters
  CollectionEventList add(fn, [bool useCaptur]) {
                         ^

In effect there is more restriction on the optional parameter than there is on the required parameter. I suppose this is needed to properly support named parameters, but it did surprise me.

I am not quite done, however. When I click the "Add sweet comic" button, I get a nasty old stack trace:

Exception: NoSuchMethodException : method not found: 'get:rect'
Receiver: Instance of '_DocumentImpl@33cc944a'
Arguments: []
Stack Trace:  0. Function: 'Object.noSuchMethod' url: 'bootstrap' line:473 col:137
 1. Function: 'ModalDialog._drawBackground@48f7d03' url: 'http://localhost:3000/scripts/ModalDialog.dart' line:75 col:25
 2. Function: 'ModalDialog.show' url: 'http://localhost:3000/scripts/ModalDialog.dart' line:38 col:20
 3. Function: 'ModalDialog.set:innerHTML' url: 'http://localhost:3000/scripts/ModalDialog.dart' line:25 col:11
 4. Function: 'AddComicForm.render' url: 'http://localhost:3000/scripts/Views.AddComicForm.dart' line:29 col:32
 5. Function: 'AddComic._toggle_form@19ec418e' url: 'http://localhost:3000/scripts/Views.AddComic.dart' line:32 col:21
 6. Function: '_EventListenerListImpl@33cc944a.function' url: '/mnt/data/b/build/slave/dartium-lucid64-inc/build/src/out/Release/obj/gen/webkit/bindings/dart/html/dartium/EventTarget.dart' line:16 col:72

The rect property used to be a property on the Document object. Now it seems to be a property of DocumentElement. So, instead of writing:

  _drawBackground([_]) {
    // ...
    window.document.rect.then((document) {
      // ...
    });
  }

Now I have to write:

  _drawBackground([_]) {
    // ...
    window.document.documentElement.rect.then((document) {
      // ...
    });
  }

The additional property feels odd, but I am not going to spend too much time worrying about it tonight. With that change in place, I again have a working Dart modal dialog:

Even better, I can save and Hipter MVC is still able to make a REST-like collection update:

That will serve for a stopping point tonight. I am pleased that everything works with just a couple of minor changes. That is pretty amazing considering how fast the language is evolving.


Day #396

SPDY/3 Flow Control Comparisons

‹prev | My Chain | next›

Last week, I compared Chrome and Firefox spdy/3 implementations. The result was inconclusive because I was using a sample page too far removed from the norm. Tonight, I revisit that comparison.

I again start with an artificial round trip time (RTT) of 100ms:

➜  spdybook-site git:(master) ✗ sudo tc qdisc add dev lo root netem delay 50ms        
[sudo] password for chris: 
➜  spdybook-site git:(master) ✗ ping localhost
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=100 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=2 ttl=64 time=100 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=3 ttl=64 time=100 ms
...

It is not quite realistic, but it is close enough and has the virtue of making math easier.

As I did last night, I use a web page that includes roughly 50 smallish images and other resources (85 is typical):

For each trial, I stop the node-spdy (spdy-v3 branch) server and close the browser before making a connection. Ideally this will make for consistently cold TCP/IP pipes over which the SPDY connections will be made.

I run a regular Chrome connection over spdy/3:

Then a Chrome connection over spdy/3 with spdy server push:

And, lastly Firefox over vanilla spdy/3 (Firefox does not support push):

I had expected to find that the Firefox connection is faster than the vanilla Chrome connection. New in spdy/3 is flow control, which I thought would make the difference. But upon reflection, flow control should not enter into this—none of the images are large enough to threaten SPDY's default receive window (64kb). In other words, there should be no flow control.

So what is the difference? I suspect that the difference is that Firefox is requesting more resources initially, pushing the TCP/IP CWND up faster. This would seem to be born out by virtue of the first large packets in each starting at the 0.7 second mark for Firefox and nearly 1.0 seconds for Chrome. Last night's Speed Tracer diagram would also suggest that Chrome is processing the <head> items before requesting the images from the <body>:

Regardless, I had wanted to compare more realistic spdy/3 implementations. For that, it seems that I need to add a couple of images that push the bounds flow control past the default receive window. I add them to the bottom of the <body> to get them loading last:

Now I repeat the trials for Chrome with vanilla spdy/3:

Chrome with spdy/3 and spdy server push:

And, lastly, Firefox over spdy/3:

This is what I had expected: Firefox trounces Chrome's flow control implementation. With a SPDY receive window of 256mb, Firefox effectively eschews spdy/3 flow control. The SPDY server is free to jam as much data as it wants on the wire. In this case, it completely fills the TCP/IP receive window (the top lines in the graphs).

With Chrome, on the other hand, the SPDY server is forced to wait for WINDOW_UPDATE frames before it can send back the next batch of data. Since Chrome never updates the SPDY receive window by more than 32kb, the server is unable to send enough data to push the bounds of the TCP/IP receive window.

The result is stark. Firefox completes transfer in less than 2 seconds whereas Chrome is twice as slow. The SPDY push is even worse because of a pause to transfer non-pushed resources (I am not pushing Javascript and CSS).

Chrome's current implementation seems clearly sub-optimal. In spdy/2 (and in vanilla HTTP), Chrome is more than capable of handling large TCP/IP receive windows. There is no reason for Chrome to continue to request WINDOW_UDPATES of 32kb—instead Chrome should be asking for near the TCP/IP receive window's worth of data.

That said, from the server's point of view, this implementation might be regarded as beneficial. The server is able to jam the smallish resources back to the browser in short order. The larger items, which the user would likely not expect to render immediately anyway, can be sent back at a more leisurely pace—freeing up the server to respond to other SPDY sessions.

Regardless, it is not the responsibility of the client to throttle the server's response like this. The server is more than capable of doing it on its own. So I expect that Chrome's WINDOW_UPDATE algorithm will get a little more sophisticated in the near future.


Day #395

Simple Comparisons of HTTP, HTTPS, SPDY/2 and SPDY/3

‹prev | My Chain | next›

tl;dr SPDY rules speed tests.

Amazingly, the average web page includes 85 resources (per httparchive.org). My "real world" web page only has a dozen images:

So I add separate copies to the bottom of the page:

I add an artificial 100ms round trip time:

➜  ~  sudo tc qdisc add dev lo root netem delay 50ms
[sudo] password for chris: 
➜  ~  ping localhost
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=100 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=2 ttl=64 time=100 ms

That is a little higher than is typical in the wild. I like it because it makes math easier and is not too far off.

Now, when I load the vanilla SSL site and watch in Speed Tracer, I see:

A total of 5.03 seconds (5.98 - 0.995) for SSL.

The "real world" SPDY running on the spdy-v3 branch of node-spdy:

SPDY/3: 1.1 seconds.

And finally, a push version that uses SPDY server push to push all of the images directly into Chrome's chache after the first request:

SPDY/3 push: 0.74 seconds.

Since I am doing all of this comparing, I might as well see what vanilla HTTP looks like:

HTTP: 2.7 seconds.

Important the SPDY vs HTTP comparison is not a fair one. Because of some deficiencies in Speed Tracer, it is not possible to include the SSL handshake in addition to the initial SPDY connection. The SPDY connection is from a cold start (I stop the server before each). Still, the SSL handshake would add ~0.5 seconds to the entire conversation, meaning that SPDY is still an easy winner.

For completeness' sake, spdy/2 looks like:

SPDY/2: 1.25 seconds.

And SPDY/2 with push:

SPDY/2: 1.03 seconds.

So the final tally is:

Trial	Time (s)
SSL	5.03
spdy/3	1.1
spdy/3 push	0.74
HTTP	2.7
spdy/2	1.25
spdy/2 push	1.03

I am unsure why the spdy/3 numbers both beat the corresponding spdy/2 numbers. I would expect them to be about the same since spdy/3 adds flow control, but little to speed up this particular page. For now, I chalk it up to small number statistics—I only ran one run for each.

Regardless, the clear winner is SPDY. Tomorrow, I will repeat the spdy/3 part of the experiment with Chrome and Firefox—this time watching TCP traces. That should serve as a nice last word on spdy/3 flow control.

Day #394

Spdy/3 Push

‹prev | My Chain | next›

I have not had any brain waves since yesterday's failure to get SPDY server push working with version 3 of the protocol's specification. It is disappointing given that I was able to add support for it to node-spdy back when spdy/2 was the latest version and since I have been able to support the new spdy/3 flow control in node-spdy. I had hoped that SPDY server push would just work. Alas.

Before soliciting others for advice, I would like to throw up a sample site that others might use to debug. Since www.spdybook.com is already spdy/3 powered, that seems like a good place to do it.

So I go about copying over the views, images and stylesheets that make up this awesome, "real" world site:

Next, I need to copy in the express.js routes. I add one for regular access (/real) and one for the currently broken push (/push):

app.get('/real', function(req, res){
  res.render('real', {
    title: 'Hello (real) World!'
  });
});


app.get('/push', function(req, res){
  // do push stuff

  res.render('real', {
    title: 'Hello (real) World!'
  });
});

Say... while looking at the "// do push stuff" section, I notice that I never tried a colon header for status:

app.get('/push', function(req, res){
  var headers = {
    ':scheme': 'https',
    ':host': 'localhost:3000',
    // ':path': '/stylesheets/style.css',
    ':path': '/images/00-h.jpg'
    // 'content-type': 'image/jpeg'
    // status: 200,
    // version: 'http/1.1',
    // url: url,
    // ':method': 'GET',
    // ':version': 'HTTP/1.1',
    // ':last-modified': (new Date).toGMTString()
  };
  // ...
});

Odd, I thought I had tried every combination. It couldn't be that simple, could it?

YUP!

Updating the headers to include a spdy/3 colon header like so:

app.get('/push', function(req, res){
  var headers = {
    ':scheme': 'https',
    ':host': req.headers.host,
    ':status': 200,
    ':version': 'HTTP/1.1',
    ':path': '/images/00-h.jpg'
  };

  res.push('/images/00-h.jpg', {}, function(err, stream) {
    fs.createReadStream('public/images/00-h.jpg').pipe(stream);
  });
  // ...
});

Does the trick. In Chrome's SPDY tab (under chrome://net-internals), I see the SPDY session, the push packets, the adopted SYN_STREAM and, most importantly, no "invalid" stream error:

SPDY_SESSION_PUSHED_SYN_STREAM
--> associated_stream = 1
--> flags = 2
--> :host: localhost:4000
    :path: /images/00-h.jpg
    :scheme: https
    :status: 200
    :version: HTTP/1.1
--> id = 2

SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 1300
--> stream_id = 2
SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 1300
--> stream_id = 2
SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 1300
--> stream_id = 2
SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 1300
--> stream_id = 2
SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 1300
--> stream_id = 2
SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 1300
--> stream_id = 2
SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 346
--> stream_id = 2
SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 0
--> stream_id = 2

SPDY_STREAM_ADOPTED_PUSH_STREAM

I had truly despaired that I would not get that working. Turns out I only needed fresh eyes.

I push those two routes out to the SPDY Book marketing site. If you would like to try them out, they are at:

• https://www.spdybook.com/real (regular spdy/3)
• https://www.spdybook.com/push (spdy/3 with push)

Before calling it a night, I make pretty graphs with the Speed Tracer plugin for Chrome. The "real world" site:

From the start of the graph a 2.45s (I really wish Speed Tracer could normalize to 0 seconds) until the last image is transferred past the 3.16 second mark is roughly 0.7 seconds. I should probably add more images and make them a little larger to make my "real world" example more real world. Still that 0.7 seconds is nothing to sneeze at.

How about with push enabled?

From the start at the 9.32 mark until the last image is pushed into browser cache at roughly 9.62 seconds means that SPDY push moved the same already very fast site in 0.3 seconds. Nice!


Day #393

Can't SPDY/3 Push

‹prev | My Chain | next›

I have sample node-spdy sites available. The older version that still relies on express-spdy is pretty solid. The spdy-v3 branch more or less works... except for SPDY server push.

SPDY server push is a fascinating (and a little scary) feature of SPDY. It allows the server to push resources directly into the browsers cache. In this way, resources can get served without the round-trip request-response typified by old HTTP.

I try to push the stylesheet for my "real world sample site" to the browser when it requests the /real resource:

app.get('/real', function(req, res){
  headers = {
    ':scheme': 'https',
    ':host': 'localhost:3000',
    ':path': '/stylesheets/style.css'
  };

  res.push('/stylesheets/style.css', headers, function(err, stream) {
    fs.createReadStream('public/stylesheets/style.css').pipe(stream);
    // fs.readFile('public/stylesheets/style.css', function (err, data) {
    //   if (err) throw err;
    //   console.log('stream writing: ' + data);
    //   stream.end(data);
    // });
  });

  res.render('real', {
    title: 'Hello (real) World!'
  });
});

I added the colon headers (:scheme, :host, etc) since spdy/3 requires them. Then I create a readable stream from the file system and pipe it back to the push stream.

And it almost works. In the SPDY tab of chrome://net-internals, I see the push stream initiated immediately after the request is send and before the SYN_REPLY reply comes back:

SPDY_SESSION_SYN_STREAM
--> flags = 1
--> :host: localhost:3000
    :method: GET
    :path: /real
    :scheme: https
    :version: HTTP/1.1
    accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
    accept-encoding: gzip,deflate,sdch
    accept-language: en-US,en;q=0.8
    referer: https://localhost:3000/
    user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.11 Safari/536.11
--> id = 1
SPDY_SESSION_PUSHED_SYN_STREAM
--> associated_stream = 1
--> flags = 2
--> :host: localhost:3000
    :path: /stylesheets/style.css
    :scheme: https
    status: 200
    url: https://localhost:3000/stylesheets/style.css
    version: HTTP/1.1
--> id = 2
SPDY_SESSION_SYN_REPLY
--> flags = 0
--> :status: 200
    :version: HTTP/1.1
--> id = 1

I even see the push data come through and my coveted "adopted" push stream:

SPDY_SESSION_PUSHED_SYN_STREAM
--> associated_stream = 1
--> flags = 2
--> :host: localhost:3000
    :path: /stylesheets/style.css
    :scheme: https
    status: 200
    url: https://localhost:3000/stylesheets/style.css
    version: HTTP/1.1
--> id = 2
SPDY_SESSION_SYN_REPLY
--> flags = 0
--> :status: 200
    :version: HTTP/1.1
--> id = 1
SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 221
--> stream_id = 2
...
SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 0
--> stream_id = 2
SPDY_STREAM_ADOPTED_PUSH_STREAM

But the page does not actually load (at least not for a minute or so). And I see this in the SPDY tab:

...
SPDY_SESSION_RST_STREAM
--> description = ""
--> status = 2
--> stream_id = 3
SPDY_SESSION_CLOSE
--> description = "bytes_read is <= 0."
--> status = -100
SPDY_SESSION_POOL_REMOVE_SESSION
--> source_dependency = 226371 (SPDY_SESSION)
SPDY_STREAM_ERROR
--> description = "ABANDONED (stream_id=2): /stylesheets/style.css"
--> status = -100
--> stream_id = 2

Back when I was first exploring SPDY server push, I found arcane rules that were needed in order get Chrome to handle the push properly. I try various combinations of those rules and attempt to push different resources, but to no avail. Even if I remove all headers but those explicitly mentioned in version 3 of the SPDY specification:

SPDY_SESSION_PUSHED_SYN_STREAM
--> associated_stream = 1
--> flags = 2
--> :host: localhost:3000
    :path: /images/00-h.jpg
    :scheme: https
--> id = 2

I still end up with an invalid stream:

SPDY_STREAM_ERROR
--> description = "ABANDONED (stream_id=2): /images/00-h.jpg"
--> status = -100
--> stream_id = 2

Looking through the Chromium source code, this error arises when a SPDY session times out and there are still streams open.

But why is that stream still open? Node-spdy sent a DATA FIN packet. Chrome even seemed to acknowledge that it received the DATA FIN by virtue of the zero length DATA packet (Chrome has never logged FIN flags for some reason) and the SPDY_STREAM_ADOPTED_PUSH_STREAM log message.

But, sure enough, after the session times out with the push stream abandoned, Chrome makes another, non-push request for my push resource in a new SPDY session:

SPDY_SESSION_SYN_STREAM
--> flags = 1
--> :host: localhost:3000
    :method: GET
    :path: /images/00-h.jpg
    :scheme: https
    :version: HTTP/1.1
    accept: */*
    accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
    accept-encoding: gzip,deflate,sdch
    accept-language: en-US,en;q=0.8
    referer: https://localhost:3000/real
    user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.11 Safari/536.11
--> id = 1

In other words, even though I said that I was going to push that image, Chrome seemed to acknowledge that I was pushing that image and that it received bunches of data without error, this resource is still not in Chrome's browser cache.

I suspect that I am missing a magic header or two somewhere. I will likely have to ask on the mailing list because I have to call it a night with it still broken.


Day #392

Sample Node-Spdy Sites (from The SPDY Book)

‹prev | My Chain | next›

I am nearly done with my exploration of SPDY flow control. New in version 3 of the protocol, flow control has proven tough to pin down, but I think I have a good handle on it.

One loose end is a more realistic example of a web page. I had that last year and it turns out that some readers of The SPDY Book want to play with that code. So why not kill two birds with one stone?

I git init a new repository and copy over the express-spdy code. One change that I need is to edit the package.json. The following should allow someone with a recent stable node.js (i.e. node.js 0.6.x) to npm install and then run the server:

{
    "name": "spdy-sample"
  , "version": "0.0.1"
  , "private": true
  , "dependencies": {
      "express-spdy": "0.1.3"
    , "jade": ">= 0.0.1"
  }
}

So, I try it :

➜  spdy-sample git:(master) ✗ npm install
...
jade@0.26.0 ./node_modules/jade
├── commander@0.5.2
└── mkdirp@0.3.0

express-spdy@0.1.3 ./node_modules/express-spdy
├── mkdirp@0.3.0
├── express@2.5.9 (qs@0.4.2, mime@1.2.4, connect@1.8.7)
├── connect-spdy@0.1.2 (connect@1.8.7)
└── spdy@0.1.4 (zlibcontext@1.0.9)
➜  spdy-sample git:(master) ✗ node app
Express server listening on port 3000

Nice!

Loading the page up in the browser, I see a super cool SPDY page:

The SPDY tab in chrome://net-internal verifies not only that this is SPDY (version 2), but also that SPDY server push still works:

SPDY_SESSION_SYN_STREAM
--> flags = 1
--> accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
    accept-encoding: gzip,deflate,sdch
    accept-language: en-US,en;q=0.8
    cache-control: no-cache
    cookie: [value was stripped]
    host: localhost:3000
    method: GET
    pragma: no-cache
    referer: https://localhost:3000/
    scheme: https
    url: /real
    user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.11 Safari/536.11
    version: HTTP/1.1
--> id = 1
SPDY_SESSION_SYN_REPLY
--> flags = 0
--> connection: keep-alive
    content-length: 3316
    content-type: text/html
    status: 200 OK
    version: HTTP/1.1
    x-powered-by: Express
--> id = 1
SPDY_SESSION_PUSHED_SYN_STREAM
--> associated_stream = 1
--> flags = 2
--> last-modified: Sun, 20 May 2012 03:11:40 GMT
    status: 200
    url: https://localhost:3000/stylesheets/style.css
    version: http/1.1
--> id = 2
....
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
SPDY_STREAM_ADOPTED_PUSH_STREAM
...

The SPDY_STREAM_ADOPTED_PUSH_STREAM is important—unless you see that, something went wrong with SPDY push.

I add a README and the sample app is ready on GitHub. For good measure, I add a separate spdy-v3 branch as well.

Hopefully this will prove useful to folks.


Day #391

SPDY-ize a Real Site for $20

‹prev | My Chain | next›

I've been (rightfully) given my share of grief for not SPDY-izing spybook.com. Well that stops today!

I have spdybook.com on DNSimple (which I highly recommend). So I request an SSL certificate through them:

I do not do the certificate signing request myself—anything to avoid actual work. I opt for a "www" certificate because it is cheap and because it will work for both www.spdybook.com and spdybook.com (I will just worry about the former tonight).

After purchasing the certificate, I choose a contact:

And then I am ready:

At this point my certificate is processing:

A little while later, I receive an email from RapidSSL asking for approval for the certificate request. Following the link in the email, all that I have to do is press the "I Approve" button a the bottom of the page:

With that, I have two certificates on DNSimple: "Private Key" and "Certificate". I save the contents of "Private Key" as private_key.pem on my Linode (which I also highly recommend). I save "Certificate" in a file named cert.pem. Lastly, I save the certificate authority's certificate (linked by DNSimple) in ca.pem.

I could use express-spdy to serve this up, but... what the hell? I might as well be bleeding edge and serve up spdy/3 from the spdy-v3 branch of node-spdy (instructions). I configure the express.js server to use my certificates:

var express = require('express')
  , spdy = require('spdy')
  , fs = require('fs')
  , routes = require('./routes');

var options = {
  key: fs.readFileSync(__dirname + '/keys/private_key.pem'),
  cert: fs.readFileSync(__dirname + '/keys/cert.pem'),
  ca: fs.readFileSync(__dirname + '/keys/ca.pem')
};
// ...

The web site itself is static (well, compiled from jekyll), so, after pointing my load balancer at the new server, I have an SSL enabled version of The SPDY Book site:

And, more importantly, I can check the SPDY tab in chrome://net-internals to verify that I am, in fact, serving up spdy/3 on https://www.spdybook.com:

Yay!

I still need to move spdybook.com over here as well (and add the port 80 redirect). But this will serve as a good stopping point for tonight.


Day #390