Monday, July 18, 2011

BDD: Book Driven Development

‹prev | My Chain | next›

While working through one of the chapters on SPDY server push in SPDY Book, I think I finally settled on the API that I want for this sort of thing in express-spdy.

This is how I'd like it to read:
var app = module.exports = express.createServer({
key: fs.readFileSync(__dirname + '/keys/spdy-key.pem'),
cert: fs.readFileSync(__dirname + '/keys/spdy-cert.pem'),
ca: fs.readFileSync(__dirname + '/keys/spdy-csr.pem'),
NPNProtocols: ['spdy/2'],
push: simplePush
});
For the push callback, I would like something like:
function simplePush(pusher) {
// Only push in response to the first request
if (pusher.streamID > 1) return;

// Push CSS data out before the web page response
pusher.pushFile("public/stylesheets/style.css", "https://localhost:3000/stylesheets/style.css");

// Push secondary web pages out after the web page response
pusher.pushLater([
["public/one.html", "https://localhost:3000/one.html"],
["public/two.html", "https://localhost:3000/two.html"],
["public/three.html", "https://localhost:3000/three.html"]
]);
}
Essentially, I want the callback to instruct the response what to push before the response (pushFirst) and what to push after the response (pushLater). I think I would like to make push an alias for pushLater because, more often than not, pushing resources after the response has been sent is the right thing to do.

The pushFirst method is already in place. I just need to get pushLater working.

To do so, I loop over each resource passed into pushLater() and initialize a push-stream:
Response.prototype.pushLater = function(resources) {
var that = this;

this.deferred_streams = [];

// Send headers for each post-response server push stream, but DO
// NOT sent data yet
resources.forEach(function(push_contents) {
var filename = push_contents[0]
, url = push_contents[1]
, data = fs.readFileSync(filename)
, push_stream = createPushStream(that.cframe, that.c, url);

push_stream._flushHead();
push_stream._written = true;
that.deferred_streams.push([push_stream, data]);
});
};
I only flush the header of the push stream to the browser. The data gets squirreled away, along with a reference to the push streams so that the data can be sent after the original response is handled. The push stream headers need to be sent before the original response is closed, but the data can be deferred until later.

Along with pushLater, I create a _pushDataLater which pushes the actual data after the response is sent:
Response.prototype._pushLaterData = function(resources) {
if (typeof(this.deferred_streams) == 'undefined') return;

this.deferred_streams.forEach(function(stream_and_data) {
var stream = stream_and_data[0]
, data = stream_and_data[1];

stream.write(data);
stream.end();
});
};
And it works! Almost.

When I access the site, the secondary web pages are pushed. So is the CSS:
t=1311041561541 [st=  197]     SPDY_SESSION_PUSHED_SYN_STREAM  
--> associated_stream = 1
--> flags = 2
--> content-type: text/css
status: 200
url: https://localhost:3000/stylesheets/style.css
version: http/1.1
--> id = 2
...
t=1311041561841 [st= 497] SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 254
--> stream_id = 2
t=1311041561841 [st= 497] SPDY_SESSION_RECV_DATA
--> flags = 0
--> size = 0
--> stream_id = 2
...
But, when I request the next page on the site, which uses the same CSS, it is requesting the CSS again:
...
t=1311041603258 [st=41914] SPDY_SESSION_SYN_STREAM
--> flags = 1
--> accept: text/css,*/*;q=0.1
accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
accept-encoding: gzip,deflate,sdch
accept-language: en-US,en;q=0.8
host: localhost:3000
method: GET
referer: https://localhost:3000/one.html
scheme: https
url: /stylesheets/style.css
user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/14.0.814.0 Safari/535.1
version: HTTP/1.1
--> id = 5
....
It is not as if server push does not work—there was no request for this HTML page. It is also not as if the CSS server push is not entirely working—when requesting the first page, there is no CSS request. Maybe this version of Chrome is buggy with respect to CSS and caching. More likely I am just missing something.

I will investigate more. Tomorrow.


Day #78

Sunday, July 17, 2011

HTTP and Slow ACKs

‹prev | My Chain | next›

I continue to play with Speed Tracer tonight. It is a wonderful visualization tool, but sometimes confuses the hell out of me.

Yesterday, I was able to create a nice, teachable diagram like the following:



It is teachable in SPDY Book because 6 images, 02-08, are downloaded in parallel. That makes it quite obvious that Chrome (and all other browsers) have a maximum of 6 interweb tubes to load resources in parallel.

Today, however, I am seeing graphs like the following:



Images 02-06 look to be downloaded around the same time. The next two images do not start returning to the browser until those first four complete. In other words, it looks like there are four, not six tubes in use.

Maybe I am obsessing over small details, but I hate the idea of not knowing a thing. That personality quirk is only magnified because I am basing content in SPDY Book on this knowledge. I must know. I will have my answer!

To solve any network related problems, I drop to Wireshark.



Hrm... looking closer at those packets, I see that immediately after requesting the first two images, Chrome is setting up the other tubes:



TCP/IP SYN packets are the first part of a TCP/IP handshake. It certainly makes sense that, if I have not visited a site before, that these tubes would have to be established.

A closer look at timing of those packets reveals that it takes a little more than 100ms for the connection to be fully established (from initial SYN, to SYN-ACK from the other side to the final ACK):



Ah… That is the 50ms delay that I have in place on this network device.

Dang it. So everything is working just fine. The first two tubes, previously established by transferring over the single CSS file and jQuery are used for the first two images. The next four images are transferred once four new tubes are established. With a 100ms round trip time, that takes a while. In the meantime, the original two tubes come free to request the next images.

That begs the question of how I managed the screenshot from yesterday. The transfers for CSS and JS were really brief. Perhaps that has something to do with it. More likely, I think, is that I had previously established a connection to the site, reloaded and, thus re-used already established tubes.

I hate to lose that teachable moment in SPDY Book, but better that than faking a connection. The six tubes will just have to be introduced as, "sometimes tubes look like this…".

Day #76

Saturday, July 16, 2011

No, Speed Tracer, No!

‹prev | My Chain | next›

Today I take a break from bashing my head against pipelining to take a closer look at Speed Tracer. Speed Tracer is a nice little Chrome extension for visualizing rendering and networking events.

For the SPDY Book, I have a little test page that includes CSS, javascript and several images. When I examine the page with Speed Tracer, I see the following in the "Network" tab:



More than one thing strikes me as odd with this trace. Given my understanding of browsers and their 6 tubes, I would expect 6 requests to start as soon as the page was parsed. But there are only two: for the CSS and Javascript on the page. I suppose that I can convince myself that Chrome requests resources inside the <head> section of a web page first. Maybe that makes sense.

But what happens next is even stranger. Chrome sends out 10 requests at the same time. As far as I know, Chrome does not use pipelining and has a maximum of 6 interweb tubes on which to request concurrent resources. But, according to Speed Tracer, Chrome is requesting 10+ images at the same time.

Stranger still, the first 2 images come back relatively quickly while the others take progressively longer:



The light blue on the left side of the request is how long it takes the request to be sent and the time waiting for the response to start coming back. All requests in this diagram are at least 100ms because there is a 100ms round trip time between the client and server. The response itself takes a while because the image is largish (10kb) and the interweb tube being used is not warm.

After those first two requests then the next 6 requests come back a while later. All 6 return at nearly the same time. Six. As in the number of tubes being used by modern browsers.

To figure this out, I drop down to Wireshark to have a look at the packets. Looking at the end of the response for the Javascript and CSS, I find:



The next 6 images are being requested at almost the same time. But wait a second... Speed Tracer told me that all 10+ images were being requested at the same time. Wireshark, which I definitely believe, is telling me that only 6 images are requested and then nothing happens other than data responses.

What the hell, Speed Tracer… What the hell?

Scrolling down a bit, once the first 6 images are sent back, only then does Chrome start the next requests:



So Chrome is definitely not pipelining. All images are requested on separated interweb tubes. There are six tubes. Just like I expected.

So Speed Tracer is lying. Great.

Even better, I notice the following request going out before the initial request for the web resource:



What the heck is a /symbolmanifest? I certainly did not put that in my sample application.

Ugh. It turns out to be a Speed Tracer mechanism for de-obfuscating Javascript. I suppose that might help Speed Tracer to analyze some of what the Javascript is doing on a page, but... it is loading before the rest of the application. After that response, the actual page is loaded—on the same tube. The tube is getting pre-warmed by a request that is hidden from the user by a tool that is supposed to help analyze, among other things, network issues.

Bah! Bad form Speed Tracer. Bad form.

I'll open an issue on the Speed Trace tracker about this. In the meantime, it seems I need another tool to help with SPDY Book.

Ugh.


Day #74

Friday, July 15, 2011

Firefox Pipelining is Silly

‹prev | My Chain | next›

After a couple of days investigation, I realized that my experiment with pipelining in Firefox was actually perfectly crafted. Perfectly crafted for failure, that is.

My pipeline.html page referenced 10 images, each around 150 kb in size. I had hypothesized that, after loading a start page, Firefox would attempt to load pipeline.html and all 10 images in the same pipeline. I was wrong. After loading pipeline.html, Firefox downloaded 6 of the images in 6 different interweb tubes (all modern browser have 6 interweb tubes to load resources in parallel). When the first of those images completed, Firefox then proceeded to download the next 4 images over a single tube.

Given that those next 4 images, pipelined or not, were 600 kb in total, it took nearly a full second to download them over a single tube. Ultimately, pipelining was a significant loser over just using the normal 6 tubes.

I am unsure why Firefox choose this strategy. It already knows that the server can handle HTTP/1.1, why not try pipelining right away? Maybe it wants to ensure that it loads enough data to appear responsive before attempting something out of the norm?

If that is the case, then once Firefox knows that the server supports HTTP pipelining, surely it will use pipelining for all subsequent requests. Using a single tube will allow its TCP/IP congestion window (CWND) to quickly grow to a healthy maximum between the client and server since it would not have to compete with 5 other tubes. Large CWND means more data for each round trip, which means fewer round trips, which means speed (this is why SPDY uses a single tube for everything).

To test this hypothesis, I use a third page after pipeline.html. On this third page, I use smaller images (~10 kb each) to better represent normal web pages. Clicking on this page after pipeline.html should re-use the HTTP tube that ended up pipelining the last four images in pipeline.html.

After restarting Firefox and clearing my cache, I access the start page, the pipeline page and my new page (a hello world from SPDY Book). Checking things out with Wireshark, I find that the same four images from pipeline.html are still pipelined:



But, when I access the next page, Firefox goes back to its old ways. Specifically, it tosses all CWND goodness from the previously pipelined tube and makes 6 different connections on 6 different tubes:



When the first of those images are downloaded, Firefox again starts pipelining. The remaining 5 images from the final page are then loaded on the same tube.



That seems like a terrible strategy. First create 6 tubes that compete with each other for full access to the bandwidth between client and server, which does not allow any to quickly increase the CWND. Then pick one tube and ramp it up with a burst of data. With that tube properly warmed, then ditch it to use the other 5 tubes again, some of which may not be warmed enough to transfer data very fast. What's worse is that, because those 6 tubes are again competing with each other, the CWND may decrease because there is congestion going on. And finally, switch back to pipelining now that the 6 tubes are just getting warmed up.

Maybe there is some reason to this seeming madness. If so, I will try to dig it up tomorrow. For now, back to putting that hello world page to good use—in SPDY Book!


Day #74

Thursday, July 14, 2011

Firefox Pipelining: Be Afraid

‹prev | My Chain | next›

Continuing my adventures with Firefox and HTTP/1.1 pipelining (a partial SPDY competitor), I begin analysis of the sessions that I have captured the past couple of nights.

Last night, I was finally able to get pipelining working in Firefox (or at least confirm that I had it working). My test page contains:
<p><img src="pipeline01.png"/></p>
<p><img src="pipeline02.png"/></p>
<p><img src="pipeline03.png"/></p>
<p><img src="pipeline04.png"/></p>
<p><img src="pipeline05.png"/></p>
<p><img src="pipeline06.png"/></p>
<p><img src="pipeline07.png"/></p>
<p><img src="pipeline08.png"/></p>
<p><img src="pipeline09.png"/></p>
<p><img src="pipeline10.png"/></p>
Each of those images are ~150kb. That's more than a typical web image ought to be, but it ought to really push pipelining and TCP/IP congestion window (CWND) through their respective paces. So let's have a look-see.

The tcptrace tool serves the dual purpose of identifying the most heavily used TCP/IP pipes (in bold below) and generating xplot graphs:
➜  14 git:(master) tcptrace -S -n -zx -zy  ../13/firefox_pipeline.pcap
1 arg remaining, starting with '../13/firefox_pipeline.pcap'
Ostermann's tcptrace -- version 6.6.7 -- Thu Nov 4, 2004

240 packets seen, 240 TCP packets traced
elapsed wallclock time: 0:00:00.033535, 7156 pkts/sec analyzed
trace file elapsed time: 0:01:29.060533
TCP connection info:
1: :0001:60110 - :0001:80 (a2b) 1> 1< (reset)
2: 127.0.0.1:48979 - 127.0.0.1:80 (c2d) 12> 8< (complete)
3: :0001:49647 - :0001:80 (e2f) 1> 1< (reset)
4: 127.0.0.1:55785 - 127.0.0.1:80 (g2h) 33> 36< (complete)
5: :0001:49649 - :0001:80 (i2j) 1> 1< (reset)
6: :0001:49650 - :0001:80 (k2l) 1> 1< (reset)
7: :0001:49651 - :0001:80 (m2n) 1> 1< (reset)
8: :0001:49652 - :0001:80 (o2p) 1> 1< (reset)
9: :0001:49653 - :0001:80 (q2r) 1> 1< (reset)
10: 127.0.0.1:55791 - 127.0.0.1:80 (s2t) 13> 13< (complete)
11: 127.0.0.1:55792 - 127.0.0.1:80 (u2v) 13> 13< (complete)
12: 127.0.0.1:55793 - 127.0.0.1:80 (w2x) 12> 13< (complete)
13: 127.0.0.1:55794 - 127.0.0.1:80 (y2z) 12> 13< (complete)
14: 127.0.0.1:55795 - 127.0.0.1:80 (aa2ab) 12> 13< (complete)
15: 127.0.0.1:55796 - 127.0.0.1:80 (ac2ad) 6> 4< (complete) (reset)
The pipe with the most activity, designated g2h by tcptrace, is the one that pipelined the last four requests (for pipeline07.png, pipeline08.png, pipeline09.png, and pipeline10.png) into a single request. Checking the plot of those packets, I find:


(the distance between the segments comes from a 100ms round trip time)

Looking at that graph, two things stick out for me: (1) it takes a long time to transfer all of that data (well over a second) and (2) the CWND (blue bar at the top) is quite variable. With regards to the CWND in (2), I am used to seeing a CWND grow relatively steadily as the interweb tube warms (a.k.a. TCP/IP slow start kicks in) when using SPDY. That is clearly not happening here. At points, the CWND shrinks, limiting the amount of data that can be transferred before waiting until the next trip. That ultimately adds time to the transfer.

But I don't think that explains the full delay. Looking at a similar diagram from a non-pipelined Firefox request (from two nights ago prior to enabling pipelining), I find:



There is similar choppiness in the CWND, but... ah wait. I see that I do not have a direct comparison. The non-pipelined session had a only a little more than 30,000 TCP/IP sequences. The pipelined request saw 80,000. Zooming in on the first 30,000 sequences of the pipelined session, I see:



That is almost identical to the non-pipelined version. Both have about the same choppiness and take ~600ms to run through the same number of sequences. That is to be expected, after all they are both non-pipelined up until this point. So it seems that everything is more or less OK until pipelining kicks.

When HTTP pipelining does start, it has a negative impact. By the time the pipelining session kicks in, the 6 TCP/IP tubes are still warming up and likely competing with each other for resources. The single tube that gets the pipeline is also still warming. Slamming it with 4 150kb images only helps so much with the CWND (again, possibly due to the other 5 open sessions). The end result is that pipelining is a big loser—almost doubling the time needed to serve up the page. Serving those 4 images over 4 different tubes would have been a better choice in this case.

I suppose edge cases like this (150 kb images) are why Firefox does not have pipelining on by default. I think tomorrow I will investigate a more realistic web case to see how the results change. Regardless, single tube SPDY connections will certainly handle something like this. That's something to prove another day as well.

Day #73

Wednesday, July 13, 2011

Firefox Pipelined!

‹prev | My Chain | next›

Last night I unsuccessfully tried to explore HTTP pipelining in Firefox. Pipelining, the sending multiple requests before the web server has replied to the first request, is major feature of SPDY. Pipelining is actually a feature of HTTP/1.1 only no one supports it.

That's not 100% true. Opera supports it, though only under certain circumstances. Currently Firefox only supports it with an about:config change. Still, there is active work with pipelining ongoing in Firefox so it seemed worth checking out.

Except it did not work.

That seemed crazy to me. There are all sort of discussions about Firefox and pipelining to be found. How could it simply not work?

So I did a bit more research and finally read the second sentence of the fifth paragraph in Mozilla's HTTP/1.1 Pipelining FAQ:
We also should not pipeline requests on a new connection, since it has not yet been determined if the origin server (or proxy) supports HTTP/1.1. Hence, pipelining can only be done when reusing an existing keep-alive connection.
Aaahh...

That's not quite how RFC 2616 reads:
Clients which assume persistent connections and pipeline immediately after connection establishment SHOULD be prepared to retry their connection if the first pipelined attempt fails.
But what the hey? I can still test that.

I create a pre-pipelining page. After accessing pre-pipeline.html and then waiting for a 5-count, a subsequent request of pipeline.html should pipeline that resource and everything referenced inside:
<p><img src="pipeline01.png"/></p>
<p><img src="pipeline02.png"/></p>
<p><img src="pipeline03.png"/></p>
<p><img src="pipeline04.png"/></p>
<p><img src="pipeline05.png"/></p>
<p><img src="pipeline06.png"/></p>
<p><img src="pipeline07.png"/></p>
<p><img src="pipeline08.png"/></p>
<p><img src="pipeline09.png"/></p>
<p><img src="pipeline10.png"/></p>
They should be pipelined, only they are not. Firefox requests pipeline01.png only after the server's response for pipeline.html is complete as seen in Wireshark:



And the requests for pipeline02.png - pipeline06.png go out on separate interweb tubes (as evidenced by the ACKs to different ports):



Finally, Firefox does not issue a request for pipeline07.png until it has fully received one of the other images:


(The response is complete at the top, followed by additional packets for the other 5 images in transit, and finally the request for pipeline07.png is at the bottom)

Presumably it does the same for pipeline08.png, pipeline09.png, and pipeline10.png as well. Except...

I cannot find those requests. I examine each of the packets after the request for pipeline07.png individually. Twice. And do not see the requests for any of those images.

Eventually, I look closer a the request for pipeline07.png... And notice the request for pipeline08.png right next to it in the same packet! Scrolling through the contents of the packet I see a GET for pipeline07.png all the way through pipeline10.png:



So Firefox does pipeline after all. It just takes a really long time for it to kick-in. Well that's not exactly competing with SPDY for fast page load times, but it should definitely help subsequent resource loads.

I will pick back up tomorrow by analyzing the eventual-pipelined stream. For now, I'm off to write some more of SPDY Book!

Day #72

Tuesday, July 12, 2011

Firefox Pipelined?

‹prev | My Chain | next›

Due to time constraints related to my impending deadline for SPDY Book, I need to move from exploring alternative protocols to SPDY to alternative approaches.

A major feature of SPDY is pipelining—using a single interweb tube for all communication. Non-pipelined HTTP requests block an HTTP tube until the server completes sending a response. Only once the entire response has been sent can a second HTTP request go out. As you can imagine, this is terribly inefficient. All major browsers get around this by opening not one, but 6 interweb tubes to a single web server. But even those tubes block waiting on responses, which means that, if you need more than 6 resources, something is going to block.

It turns out that HTTP/1.1 actually has pipelining built in—only you wouldn't know it based on browser support. Opera is rumored to support it in certain, unrealistic edge cases, but that is it. Except...

Firefox supports it through an about:config setting.

But first, I create a dummy pipeline.html that loads in 10 pipelineXX.png images:
<p><img src="pipeline01.png"/></p>
<p><img src="pipeline02.png"/></p>
<p><img src="pipeline03.png"/></p>
<p><img src="pipeline04.png"/></p>
<p><img src="pipeline05.png"/></p>
<p><img src="pipeline06.png"/></p>
<p><img src="pipeline07.png"/></p>
<p><img src="pipeline08.png"/></p>
<p><img src="pipeline09.png"/></p>
<p><img src="pipeline10.png"/></p>
The images themselves are on the large side:
➜  www  ls -lh pipeline*
-rw-r--r-- 1 root root 157K 2011-07-12 23:06 pipeline01.png
-rw-r--r-- 1 root root 157K 2011-07-12 23:06 pipeline02.png
-rw-r--r-- 1 root root 157K 2011-07-12 23:06 pipeline03.png
-rw-r--r-- 1 root root 157K 2011-07-12 23:06 pipeline04.png
-rw-r--r-- 1 root root 157K 2011-07-12 23:06 pipeline05.png
-rw-r--r-- 1 root root 157K 2011-07-12 23:06 pipeline06.png
-rw-r--r-- 1 root root 157K 2011-07-12 23:07 pipeline07.png
-rw-r--r-- 1 root root 157K 2011-07-12 23:07 pipeline08.png
-rw-r--r-- 1 root root 157K 2011-07-12 23:07 pipeline09.png
-rw-r--r-- 1 root root 157K 2011-07-12 23:07 pipeline10.png
-rw-r--r-- 1 cstrom cstrom 351 2011-07-12 23:05 pipeline.html
To get a somewhat real-world feel for the interaction, I add a 50ms delay on my local network device:
➜  www  ping localhost
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=0.051 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=2 ttl=64 time=0.022 ms
^C
--- localhost.localdomain ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.022/0.036/0.051/0.015 ms
➜ www sudo tc qdisc add dev lo root netem delay 50ms
➜ www ping localhost
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=100 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=2 ttl=64 time=100 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=3 ttl=64 time=100 ms
^C
--- localhost.localdomain ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 100.164/100.197/100.226/0.025 ms
I load the page up in Firefox and capture all of the traffic in Wireshark. I save the filtered tcp.port == 80 traffic into a pcap which I analyze with tcptrace:
➜  not_pipelined git:(master) ✗ tcptrace -S -n -zx -zy  ../firefox_not_pipelined.pcap 
1 arg remaining, starting with '../firefox_not_pipelined.pcap'
Ostermann's tcptrace -- version 6.6.7 -- Thu Nov 4, 2004

246 packets seen, 246 TCP packets traced
elapsed wallclock time: 0:00:00.024607, 9997 pkts/sec analyzed
trace file elapsed time: 0:00:17.608415
TCP connection info:
1: :0001:58463 - :0001:80 (a2b) 1> 1< (reset)
2: 127.0.0.1:54606 - 127.0.0.1:80 (c2d) 21> 23< (complete)
3: :0001:58465 - :0001:80 (e2f) 1> 1< (reset)
4: :0001:58466 - :0001:80 (g2h) 1> 1< (reset)
5: :0001:58467 - :0001:80 (i2j) 1> 1< (reset)
6: :0001:58468 - :0001:80 (k2l) 1> 1< (reset)
7: :0001:58469 - :0001:80 (m2n) 1> 1< (reset)
8: 127.0.0.1:54612 - 127.0.0.1:80 (o2p) 15> 14< (complete)
9: 127.0.0.1:54613 - 127.0.0.1:80 (q2r) 15> 15< (complete)
10: 127.0.0.1:54614 - 127.0.0.1:80 (s2t) 16> 21< (complete)
11: 127.0.0.1:54615 - 127.0.0.1:80 (u2v) 16> 21< (complete)
12: 127.0.0.1:54616 - 127.0.0.1:80 (w2x) 16> 21< (complete)
13: 127.0.0.1:54617 - 127.0.0.1:80 (y2z) 6> 4< (complete) (reset)
14: 127.0.0.1:54618 - 127.0.0.1:80 (aa2ab) 6> 4< (complete) (reset)
Perfect. I do not even need to look at the graphs to see that all 6 interweb tubes were in action here (tubes with more than 4 response packets).

To enable pipelining in Firefox, I enter about:config in the address bar:



The default pipelining settings are:



I enable pipelining and set the maximum number of requests to 8:



I believe that 8 is the recommended number of max-requests.

I clear the browser cache, an re-request the pipeline page. Looking at the output with tcptrace, I see no change:
➜  pipelined git:(master) ✗ tcptrace -S -n -zx -zy  ../firefox_pipelined.pcap    
1 arg remaining, starting with '../firefox_pipelined.pcap'
Ostermann's tcptrace -- version 6.6.7 -- Thu Nov 4, 2004

223 packets seen, 223 TCP packets traced
elapsed wallclock time: 0:00:00.027996, 7965 pkts/sec analyzed
trace file elapsed time: 0:00:18.368731
TCP connection info:
1: :0001:47563 - :0001:80 (a2b) 1> 1< (reset)
2: 127.0.0.1:39393 - 127.0.0.1:80 (c2d) 32> 37< (complete)
3: :0001:47565 - :0001:80 (e2f) 1> 1< (reset)
4: :0001:47566 - :0001:80 (g2h) 1> 1< (reset)
5: :0001:47567 - :0001:80 (i2j) 1> 1< (reset)
6: :0001:47568 - :0001:80 (k2l) 1> 1< (reset)
7: :0001:47569 - :0001:80 (m2n) 1> 1< (reset)
8: 127.0.0.1:39399 - 127.0.0.1:80 (o2p) 13> 14< (complete)
9: 127.0.0.1:39400 - 127.0.0.1:80 (q2r) 13> 14< (complete)
10: 127.0.0.1:39401 - 127.0.0.1:80 (s2t) 13> 14< (complete)
11: 127.0.0.1:39402 - 127.0.0.1:80 (u2v) 13> 13< (complete)
12: 127.0.0.1:39403 - 127.0.0.1:80 (w2x) 13> 12< (complete)
13: 127.0.0.1:39404 - 127.0.0.1:80 (y2z) 6> 4< (complete) (reset)
Dang it. All 6 interweb tubes are still in use.

Checking it out in Wireshark, it really seems like nothing has changed. The browser still waits until it is done processing pipeline02.png before requesting pipeline07.png:



Hrm... I'm at a bit of a loss. Maybe Firefox 5's pipelined support is lacking or maybe I am just missing something. I'll see if I can figure out which is the case. Tomorrow.


Day #72