Stream IDs and Server Push

‹prev | My Chain | next›

Up today, I explore what happens when multiple clients connect to a SPDY server. My specific goal is to understand if stream IDs need to increase across all connections or just within a single client-server session.

Per the SPDY spec, stream IDs are 31 bit integers. 2^31 = 2,147,483,648 possible stream IDs, but it is actually half that number because client initiated IDs are odd and server initiated streams have even stream IDs. That means that there are little over a billion stream frames possible before the limit is reached. Per the specification, if the limit is reached, then no more frames may be sent (wrapping IDs is not allowed).

Were stream IDs shared across all clients, 1 billion frames could be exhausted relatively quickly on a large site. Before I overthink this too much, I try things out in Chrome, checking IDs in the SPDY tab of about:net-internals. After loading the site in three separate incognito window, I find this in the third:

(P) t=1308188276893 [st=    1]     SPDY_SESSION_SYN_STREAM  
                                   --> flags = 1
                                   --> accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
                                       accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
                                       accept-encoding: gzip,deflate,sdch
                                       accept-language: en-US,en;q=0.8
                                       host: localhost:8081
                                       method: GET
                                       scheme: https
                                       url: /
                                       user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.13 Safari/535.1
                                       version: HTTP/1.1
                                   --> id = 1

Dang, er.... Cool! It just works. So node-spdy already works as desired and I do not have to change anything. I kinda like it when I have to investigate and get something working, but I suppose the occasional it just works is OK (and with node-spdy that happens more often than not).

Oh wait. I'm an idiot. The clients initiate the stream IDs. They have no idea what IDs the server may have already used. New connections will always be started with an ID of 1. The server just replies on the same stream. Duh.

But what about the push streams that I added to node-spdy? I have the feeling I may have made a mistake there...

I start up the test server for push and access the server in a couple of incognito windows. Again in the SPDY tab, I find:

t=1308190020006 [st=    23]     SPDY_SESSION_PUSHED_SYN_STREAM  
                                --> associated_stream = 1
                                --> flags = 2
                                --> status: 200
                                    url: https://localhost:8082/style.css
                                    version: http/1.1
                                --> id = 6

Ugh. I did make a mistake there. The stream ID should start at 2 for each client request. Checking out the code I find a TODO note I left myself:

var //...
    streamID = 0;

/**
 * Class constructor
 */
var PushStream = exports.PushStream = function(associated_cframe, c, url) {
  stream.Stream.call(this);
  this.streamID = streamID += 2; // TODO auto-increment per-stream per: http://www.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2#TOC-Stream-creation
//...
});

Ugh. That is entirely my doing. I am using a module variable (little more than a global in the PushStream module) to hold the last ID used to push stream. With each new push stream, I increment by 2 (again, all server-initiated streams must have an even stream ID). But that number keeps increasing across all pushes to all clients. Eventually I am going to run out of IDs for a heavily used server.

What if I reset the stream ID? The SPDY spec says:

It is illegal for the stream-id to not increase with each new stream. If an endpoint receives a SYN_STREAM with a stream id which is less than any previously received SYN_STREAM, it MUST issue a session error (Section 2.4.1) with the status PROTOCOL_ERROR.

A session error will drop the entire SPDY session. The client would have to re-try the request on a new stream.

I suppose that if that only happens to 1 in a billion requests it is not a huge inconvenience. But will Chrome automatically re-try or is additional coding required?

In the PushStream class, I reset the streamID after pushing out a stream ID of 6:

  if (streamID > 5)
    streamID = 0;

That should reset the stream ID in between push streams on the same request. I reload the test app three times, then check the SPDY tab in about:net-internals:

t=1308195089291 [st= 11]     SPDY_SESSION_PUSHED_SYN_STREAM  
                             --> associated_stream = 1
                             --> flags = 2
                             --> status: 200
                                 url: https://localhost:8082/style.css
                                 version: http/1.1
                             --> id = 6
...

t=1308195089292 [st= 12]     SPDY_SESSION_PUSHED_SYN_STREAM  
                             --> associated_stream = 1
                             --> flags = 2
                             --> status: 200
                                 url: https://localhost:8082/spdy.jpg
                                 version: http/1.1
                             --> id = 2

...

t=1308195089350 [st= 70]     SPDY_STREAM_ADOPTED_PUSH_STREAM  

Well, that is not right. I very clearly violated the spec by sending a stream with ID of 6 followed by a stream with a lower ID. And yet, the stream was adopted as if no problem occurred. What's more, the client never made a request for the pushed data.

Regardless of whether or not Chrome will automatically re-try a request after a SPDY session was terminated, this global sessionID is not the right solution. The streamID should start at 2 for each different client connected to the server.

It is easy enough to start with a stream ID of 2 for each new connection, but how can I increment the stream ID on subsequent requests? I do not think this is possible.

Consider Bob requests the homepage from behind a NAT. Behind the same NAT, Alice accesses a paywall section of the same site. Both requests push different resources back to the client. Then Bob tries to access the same paywall content.

In this case, the SPDY server should increment Bob's push stream ID, but how can it associate Bob's paywall request with his initial access to know his next stream ID? The request will only contain information such as:

t=1308195058479 [st= 1]     SPDY_SESSION_SYN_STREAM  
                            --> flags = 1
                            --> accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
                                accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
                                accept-encoding: gzip,deflate,sdch
                                accept-language: en-US,en;q=0.8
                                cache-control: max-age=0
                                host: localhost:8082
                                method: GET
                                scheme: https
                                url: /paywall/
                                user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.13 Safari/535.1
                                version: HTTP/1.1
                            --> id = 131

Nothing can uniquely identify Bob's session as the one that started with his homepage access. If the server attempts to to tie access by IP address, then it can easily get confused by Alice's access from behind the same network.

I am at a loss at this point, so I will call it a night here. After sleeping on it I will likely hit up the spdy-dev mailing list for guidance.



Day #51