The Case of the Extra Stream Data

‹prev | My Chain | next›

I continue my exploration of the upcoming version 0.5 of fab.js. I admit that I am getting a bit bogged down here, but I believe that understanding is slowing spreading. Hopefully tonight will prove to be a break through. Last night I explored the various degenerative cases of fab.stream. That provided some insight. Tonight I take a step back to see if I can apply that knowledge.

I continue to try to get my player_status (fab) app to embed itself inside HTML template apps:

      (route, /^\/player_status/)
        ( PRE )
          ( player_status )
        () // PRE
      () // route

That is a nice, easy-to-read representation of what I want to do. For the /player_status route, the output of the player_status (fab) app should be wrapped in a <pre> tag.

That is actually pretty easy to do—if the player data were static:

function player_status(write) {
  return write("player_status 01")
  ("\n")
  ("player_status 02")
  ("\n")
  ("player_status 03")
  ("\n");
}

Here, player_status is a normal (fab) app, invoked with the write stream for the response. Invoking that stream with a string argument, "player_status 01", sends the data in the response and also returns the downstream listener in case there is more to say (i.e. "\n", "player_status 02", etc.).

With that, I get my expected 3 player statuses inside a <pre> tag:



My problem is that I need to grab the player status from CouchDB via node-couchdb. That means callbacks. Luckily, even in the infant stages of v0.5, fab.js supplies the fab.stream app to "hit the pause button" until the data is sent back.

How does this cause problems? Well, if I write multiple times to the fab.stream, I get output like this:



And HTML that ends with:

<pre>player_status 02</pre></body></html> 
</pre></body></html>player_status 03</pre></body></html> 
</pre></body></html>player_status 01</pre></body></html> 
</pre></body></html>

I get this even with simple versions without the callbacks, just multiple stream writes:

function player_status( write ) {
  return write(function(write) {
    return fab.stream(function(stream) {
      stream(write("player_status 01"));
      stream(write("\n"));
      stream(write("player_status 02"));
      stream(write("\n"));
      stream(write("player_status 03"));
      stream(write("\n"));
      stream(write());degenerative 
    });
  });
}

In all of the use-cases of fab.stream that I considered last night, I assumed that there was no queue—no initial list of data that would always be written after the explicit writes. That assumption is valid—the queue would be a second argument to fab.stream. Here, I am only supplying a single callback to be invoked—no queue.

My mistake was in not recognizing that there is a second way to build up the queue. The other means of building up the queue is through upstream data. Looking at the fab chain, there does not seem to be any upstream data:

      (route, /^\/player_status/)
        ( PRE )
          ( player_status )
        () // PRE
      () // route

The empty argument function calls should close the HTML tag (and the route) with no data, right? This is were I went wrong. The PRE (fab) app, like all HTML apps, actually creates upstream data beyond player_status. Thinking about it, it has to this in order to create the closing </pre> tag. These HTML (fab) apps write to the normal response stream, then reads from the player_status app while supplying that downstream data.

Taking a peak inside the fab.elem (fab) app, we can see where this occurs:

  function elem( name, isVoid ) {
    return function( write, obj ) {
      write = fab.concat( write )
      write = write( "<" + name );
      write = attrs( write )( obj )( ">" );

      if ( isVoid ) return write;

      return function read( arg ) {
        if ( !arguments.length ) return write( "</" + name + ">" );

        write = write.apply( undefined, arguments );
        return read;
      };
    }

If the upstream app (player_status in this case) returns data, then write that data to the response. If the upstream is done with data (as indicated by no arguments), then write the closing tag.

The end result is that every time I stream new data back, these closing tags are coming along for the ride:

<pre>player_status 02</pre></body></html> 
</pre></body></html>player_status 03</pre></body></html> 
</pre></body></html>player_status 01</pre></body></html> 
</pre></body></html>

In the end, I can get away with (fab) apps with multiple streams only if I am sure that there is no upstream data on the way. Dang.

Armed with that knowledge, I do believe that I can resolve my issue, but I will leave that for tomorrow. I need to get some sleep before Ruby DCamp tomorrow.


Day #228