Friday, June 10, 2011

SPDY Data Compression

‹prev | My Chain | next›

I continue working through little odds and ends tonight. A while back, I was unable to get SPDY data compression working with node-spdy. I chalked up the problem to Chrome dropping support (it is a feature that is likely to go away in the near future anyway).

But, as Mike Belshe pointed out in the comments, I missed that data compression get its own compression stream. Also, I was unaware that data compression streams do not get a dictionary like header streams.

To get this to work, I need to muck with node-spdy's zlib context which assumes a dictionary zlib stream:
var ZLib = exports.ZLib = function() {
if (arguments.length > 0 && arguments[0] == false) {
this.context = new ZLibContext();
else {
this.context = new ZLibContext(flatDict);
With that, I can create my own getStreamCompressor in the node-spdy Respose class:
Response.prototype.getStreamCompressor = function(streamID) {
if (this.stream_compressor)
return this.stream_compressor;

this.stream_compressor = new ZLib(false);

return this.stream_compressor;
Lastly, I ensure that I am using this non-dictionary zlib context stream and that I am generating a compressed data frame:
  var dframe = createDataFrame(this.getStreamCompressor(), {
streamID: this.streamID,
flags: fin ? enums.DATA_FLAG_FIN : enums.DATA_FLAG_COMPRESSED,
}, Buffer.isBuffer(data) ? data : new Buffer(data, encoding));
Hopefully that will be sufficient. I had convinced myself that setting the DATA_FLAG_COMPRESSED would trigger compression of the data based on:
exports.createDataFrame = function(zlib, headers, data) {
if (headers.flags & enums.DATA_FLAG_COMPRESSED) {
data = zlib.deflate(data);

But, checking out the dialog in Chrome's about:net-internals // SPDY tab, it seems that the compression is not being set:
t=1307763599828 [st=113]     SPDY_SESSION_RECV_DATA  
--> flags = 0
--> size = 8577
--> stream_id = 7
(the flags are empty)

Bummer. Before investigating, I check the packets in Wireshark just to be sure that the flags are, in fact, empty:
00 00 00 05 |C| Stream-ID (31bits) |
02 00 00 1b | Flags (8) | Length (24 bits) |
78 9c ca 48 | Data |
cd c9 c9 57 +----------------------------------+
48 2b ca cf
55 28 4e 2d 2a 4b 2d 52 04 08 00 00 00 ff ff
So the compression is being applied after all (a flag of 0x02 means DATA_COMPRESSION ins SPDY). So, uh... Yay!

I think I will continue with data compression a bit longer tomorrow. I ought to take some time to verify that the data really is compressed and see if I can measure and time / size difference between compressed and non-compressed data streams.

Day #46

1 comment:

  1. If you think this is indeed a chrome bug, leave a server running and let me point at it. We'll figure it out quickly. I believe Chrome should be able to receive compressed data frame packets just fine. Keep in mind that Chrome does not *send* them at this time.