Network Emulation in Linux

‹prev | My Chain | next›

Up tonight, I would like to play with network emulation in Linux. As I explore SPDY, I have been doing a lot of packet inspection. I have only begun to investigate network performance and even then I have been doing everything in the blackbox that is the loopback network interface (lo).

Linux offers a decent interface into network traffic configuration named tc. Tonight I am going to play with the "queuing discipline" (qdisc) used on my localhost. Specifically, I am going to use the network emulator (netem) qdisc to add an artificial round-trip time of 200ms.

Before doing anything, when I ping localhost, I get sub-tenth of a second RTTs:

➜  express-spdy-test  ping localhost
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=0.054 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=2 ttl=64 time=0.060 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=3 ttl=64 time=0.059 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=4 ttl=64 time=0.052 ms
^C
--- localhost.localdomain ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2997ms
rtt min/avg/max/mdev = 0.052/0.056/0.060/0.006 ms

If I add a 100ms delay, I find:

➜  express-spdy-test  sudo tc qdisc add dev lo root netem delay 100ms  
➜  express-spdy-test  ping localhost                                 
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=200 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=2 ttl=64 time=200 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=3 ttl=64 time=200 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=4 ttl=64 time=200 ms
^C
--- localhost.localdomain ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 200.119/200.173/200.211/0.034 ms

Cool! The netem qdisc offers some nice options to make the RTT more like real network traffice (randomize times, but not too random). I am not sure that I need that for SPDY Book, but I will not use them tonight.

With that, I take a look at the SPDY server push packets that I began investigating last night:



Compare with the graph from last night without network emulated latency:



The most obvious difference is that the network emulated packets are much less discrete than the normal loopback packets. I would imagine that this is due to more aggressive buffering under the emulator. Something to check into another day.

The other difference is that there is definitely a 200ms difference between chunks of data. The network emulator is definitely having the desired effect in that respect.

The other think that I wanted to look into was increasing the size of the files being served. The web page response, the SPDY Server pushed web pages and the CSS are all on the order of a hundred bytes or so. Not exactly realistic.

So I add some lorem ipsum text to each to get up to:

➜  express-spdy-test  ls -lh views/index.jade public/stylesheets/style.css public/*html
-rw-r--r-- 1 cstrom cstrom 49K 2011-07-05 22:47 public/one.html
-rw-r--r-- 1 cstrom cstrom 49K 2011-07-05 22:46 public/stylesheets/style.css
-rw-r--r-- 1 cstrom cstrom 49K 2011-07-05 22:47 public/three.html
-rw-r--r-- 1 cstrom cstrom 49K 2011-07-05 22:47 public/two.html
-rw-r--r-- 1 cstrom cstrom 49K 2011-07-05 22:46 views/index.jade

Now when I load up the page, I find the response + pushed data looks like:



Finally, I am seeing the advertised TCP/IP segment window show (the blue line above the packets). What this means is that, previously, the entire site (home page, secondary pages, and CSS) could fit into a network segment or two.

Here I see TCP/IP slow start kick in. Only the last segment (just past the 3 second mark) pushes the congestion window up to a point at which copious amounts of data are being sent. The lack of increase before that is unexpected. Based on my understanding of things, I would have expected the window to increase with each ACK of a segment. Grist for another post. As-is, I am fairly pleased to have begun to play around with tc.

The last thing I do tonight is remove the delay network emulator as a queue discipline:

➜  express-spdy-test  sudo tc qdisc del dev lo root
➜  express-spdy-test  ping localhost
PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=1 ttl=64 time=0.050 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=2 ttl=64 time=0.041 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=3 ttl=64 time=0.062 ms
64 bytes from localhost.localdomain (127.0.0.1): icmp_req=4 ttl=64 time=0.054 ms
^C
--- localhost.localdomain ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2997ms
rtt min/avg/max/mdev = 0.041/0.051/0.062/0.011 ms

It would very bad if my localhost "suddenly" began to run slow when I pick back up with my day job in the morning.

I will likely come back to network emulation as I work through the last couple of chapters in SPDY Book. But tomorrow, I am going to switch node-spdy / express-spdy to the just released node 0.5.


Day #66