The setup for network testing is a real pain. Stop the server, stop the browser, start the server, start a fresh Wireshark capture, start the browser, stop the capture, save, and analyze. Then change something oh-so-slightly and repeat.
To be sure, it can produce some pretty graphs, like a Chrome SPDY/2 connection from last night:
It occurs to me that I failed to grab a Firefox spdy/2 conversation last night. I ended my research frustrated with spdy/3 charts for both Chrome and Firefox, but Firefox's flow control is significantly different than Chrome's. So, for completeness's sake, I will make new Firefox charts -- for both spdy/2 and spdy/3.
As I did last night, I set the RTT for packets to 100ms to somewhat approximate real-world connections (while still retaining easy math):
➜ ~ sudo tc qdisc add dev lo root netem delay 50ms [sudo] password for chris: ➜ ~ ping localhost PING localhost.localdomain (127.0.0.1) 56(84) bytes of data. 64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=100 ms 64 bytes from localhost.localdomain (127.0.0.1): icmp_req=2 ttl=64 time=100 ms ...With that, I capture a Firefox spdy/2 conversation:
As I suspected, that is just like the Chrome spdy/2 conversation—the outgoing packets fill the TCP/IP receive window nicely, which grows nicely. This is TCP/IP slow start in action—my single SPDY interweb tube is warming as desired.
The Firefox spdy/3 conversation last night was not nearly as satisfying. The outgoing packets filled the receive window, but the receive window never grew—resulting in a very slow SPDY conversation.
I am not going to compare last night's data with tonight's so I start a new spdy/3 capture:
Wait... that looks exactly like the spdy/2 conversations. Actually, there are slight delays as node-spdy decides if SPDY/3 flow control is warranted (it never is for Firefox), but that looks almost identical. Why did I see something so different last night? Could I be making a mistake tonight?
The answer the latter question is "no" — there is no mistake. This is a spdy/3 conversation:
So it would seem that the mistake was last night. I try reproducing the experiment several more times and always see the same thing: spdy/2 and spdy/3 are essentially identical.
In fact this is what I had been hoping to see (which makes it somewhat harder to accept tonight's results). Since Firefox specifies a per-stream initial receive window of 256mb (much larger than any resource on my test page), there should be no reason for flow control to kick in. This, in turn, should mean that spdy/2 and spdy/3 are the same—at least for Firefox.
And (tonight), that is the case. But what about Chrome?
The Chrome spdy/2 conversation remains blazing fast—essentially identical to the Firefox spdy/2 and spdy/3 conversations. But maybe I made a similar mistake with the Chrome spdy/3 conversation?
And again, this does not surprise me too much.
Where the Firefox receive window seems too large to me, the Chrome receive window seems too static. If I print out what node-spdy thinks the receive window is throughout the conversation, I see:
Express server listening on port 3000 in development mode transferWindowSize(7): 62668 transferWindowSize(9): 56560 transferWindowSize(7): 65536 transferWindowSize(9): 41744 transferWindowSize(7): 60336 transferWindowSize(9): 38876 transferWindowSize(7): 65536 transferWindowSize(9): 57860 transferWindowSize(7): 61244 transferWindowSize(9): 48760 transferWindowSize(7): 50060 transferWindowSize(9): 62420 transferWindowSize(7): 61120 transferWindowSize(9): 59160 transferWindowSize(7): 50060 transferWindowSize(9): 52020 transferWindowSize(7): 61120 transferWindowSize(9): 55260 transferWindowSize(7): 50060 transferWindowSize(9): 55260 transferWindowSize(7): 60460 transferWindowSize(9): 52020 transferWindowSize(7): 50720 transferWindowSize(9): 59160 transferWindowSize(7): 60460 transferWindowSize(9): 62420 transferWindowSize(7): 50060 transferWindowSize(9): 64360 transferWindowSize(9): 52660 transferWindowSize(9): 58520 transferWindowSize(9): 52660It never goes above the initial receive window of 64kb. So, every 64kb or so, node-spdy has no choice but to pause until Chrome updates the window. It is never too long—maybe a few hundredths of seconds—but it takes long enough that node-spdy, of more specifically, node.js, notices that it has 64kb of data lying about. That will not push the TCP/IP receive window, but it's enough that node.js will drain it and send it on the wire.
I do not think this is necessarily an indictment of spdy/3 flow control or of Chrome's implementation. Rather it suggests that Chrome spdy/3 is not optimized for a web page with little else besides two overly large (~200kb each) images.
Up tomorrow, I think I need to have a closer look at a more realistic scenario—dozens of images and resources instead of just the two. But, thankfully, I can call it a night with a little less confusion.