Wednesday, September 8, 2010

Returning Write, Not Read

‹prev | My Chain | next›

Tonight I continue my exploration of the pre-release v0.5 of fab.js. Last night I got it working exactly like I wanted, but I did not understand why it worked. Since programming by coincidence is a cardinal sin of programming, I need to investigate a bit further.

To get understanding, I need to remove all my cruft and play with an absolute minimal (fab) application:
// Don't crash on errors
process.on('uncaughtException', function (err) {
console.log('Runaway exception: ' + err.stack);
});

with ( require('fab') )

( fab )

// Listen on the FAB port
( listen, 0xFAB )

( test )
()

();
I will be fiddling with the test (fab) app tonight. The simplest (fab) app, in v0.5, looks like:
function test(write) {
return function read() {
return write(" bar\n")(" baz\n");
};
}
Simple enough: the test (fab) app is called with a write function / stream. I can write data, which returns a copy of the write function to receive more data (e.g. " baz\n"), which returns another copy of itself to be returned when the read() function is invoked.

When accessing any resource on the running server, I see:
cstrom@whitefall:~$ curl http://localhost:4011/ -i
HTTP/1.1 200 OK
Connection: keep-alive
Transfer-Encoding: chunked

bar
baz
Nothing to surprising there.

The simplest form of (fab) app that I do not understand is this:
function test(write) {
return write(function(write) {
return write(" bar\n")(" baz\n");
});
}
Checking the output with this version in place, I see that it is working exactly the same as the previous version:
cstrom@whitefall:~$ curl http://localhost:4011/ -i
HTTP/1.1 200 OK
Connection: keep-alive
Transfer-Encoding: chunked

bar
baz
How is that possible?

In the canonical (fab) app example, I return a read() callback that will be invoked on incoming request and will be called with an upstream (further from the browser) response stream to decorate as it likes. In other words, I am returning a callback that will be invoked for possible upstream reading.

So how does the second example work?
function test(write) {
return write(function(write) {
return write(" bar\n")(" baz\n");
});
}
Invoking write() will always return a downstream write stream. If that downstream write stream is invoked like the previous read callback, how does that ever make it to the browser.

I cheat to get the answer. And by cheat, I raise an exception, by referencing an undeclared variable, inside the write() method's callback:
function test(write) {
return write(function(write) {
asdf;
return write(" bar\n")(" baz\n");
});
}
When I access the server with curl, I see this stacktrace in the backend:
Runaway exception: ReferenceError: asdf is not defined
at /home/cstrom/tmp/test.js:36:5
at read (/home/cstrom/.node_libraries/fab/index.js:186:18)
at drain (/home/cstrom/.node_libraries/fab/index.js:31:21)
at Server.<anonymous> (/home/cstrom/.node_libraries/fab/index.js:69:19)
at Server.emit (events:33:26)
at HTTPParser.onIncoming (http:830:10)
at HTTPParser.onHeadersComplete (http:87:31)
at Stream.ondata (http:762:22)
at IOWatcher.callback (net:494:29)
at node.js:764:9
The last stack point in (fab) proper is inside the fab.render app, which is used to stream data back to the browser:
fab.render = function( write ) {
var args = [].slice.call( arguments );

return function read( obj ) {
if ( obj && obj.apply ) {
args[ 0 ] = read;
return obj.apply( undefined, args );
}

write = write.apply( undefined, arguments );

return arguments.length ? read : write;
}
}
Ah! So by supplying the callback to write(), one defers the evaluation (via apply()) of that app until sending data back to the client. The read() function defined inside fab.render is then sent to the callback as args[0]. The callback thinks that it is writing:
function test(write) {
return write(function(write) {
return write(" bar\n")(" baz\n");
}
);
}
...and fab.render thinks that it is reading from an ordinary upstream app.

Good stuff. I believe that there is still a little more for me to wrap my brain around in the new v0.5 fabjs, but comprehension is beginning to spread.


Day #220

3 comments:

  1. chris,

    here is the simplest possible fab app:

    function( write ){ return write }

    all this does is pass the stream back to the original. next up,

    function( write ){ return write( "hey" ) }

    all this does is write "hey", and then pass the stream back to the original.

    notice how neither of these apps is interested in the rest of the chain, and return control as soon as they are called. next up,

    function( write ) {
    return function read( data ) {
    if ( !arguments.length ) return write;

    write = write( "hey" )( data )
    return read;
    }
    }

    this app actually reads the rest of the stream until (), and writes "hey" before everything it sees.

    here's where i think you're getting tripped up. check out these two functions:

    function compileTime( write ) {
    return write( Math.random().toString() );
    }

    function runTime( write ) {
    return write( function( write ) {
    return write( Math.random().toString() );
    })
    }

    the difference here is that compileTime will write a random number to the stream, but that random number will be the same for all subsequent requests.

    runTime, on the other hand, writes a function that writes a random number, which will be different for every request.

    the idea is that the root (fab) function evaluates everything once when it's called. so functions that write a value end up writing something static, and those that write functions write something that gets called on every subsequent call.

    (i'm still on the fence about whether to make this auto-run behavior explicit.)

    the key is to understand that fab.render is recursive: it'll read a stream, and any time it sees an app, it'll call the app and all the apps within until there are no functions left.

    i hope this makes sense, let me know if you have any questions!

    jed

    ReplyDelete
  2. Jed,

    Awesome, thanks! That does help to crystallize things for me.

    I definitely saw the compile time vs. run time behavior while exploring. I was actually more surprised the compile time behavior (probably because it differs from v0.4). Once I figured out how that was happening, I was OK. Knowing the why helps that much more :-)

    Dunno that I have any insight on the autorun behavior other than to say that it ought to be documented if not made explicit (but you no doubt already planned for that). Aside from that, it is hard to anticipate how it'll affect coding. I probably need to play with it more to have a useful opinion.

    Thanks again for all the help here (and with fab.stream the day before). Looking forward to see how 0.5 evolves!

    -Chris

    ReplyDelete
  3. chris,

    i think i'll try a branch taking out the auto-run, in which case the only thing the (fab) function would do is return an app that replays the stream. being able to control evaluation would probably be a win.

    well, time to hop on a flight... will be doing a lot of dev in berlin for the next two weeks, so stay tuned.

    jed

    ReplyDelete