SPDY Data Compression

‹prev | My Chain | next›

I continue working through little odds and ends tonight. A while back, I was unable to get SPDY data compression working with node-spdy. I chalked up the problem to Chrome dropping support (it is a feature that is likely to go away in the near future anyway).

But, as Mike Belshe pointed out in the comments, I missed that data compression get its own compression stream. Also, I was unaware that data compression streams do not get a dictionary like header streams.

To get this to work, I need to muck with node-spdy's zlib context which assumes a dictionary zlib stream:

var ZLib = exports.ZLib = function() {
  if (arguments.length > 0 && arguments[0] == false) {
    this.context = new ZLibContext();
  }
  else {
    this.context = new ZLibContext(flatDict);
  }
};

With that, I can create my own getStreamCompressor in the node-spdy Respose class:

Response.prototype.getStreamCompressor = function(streamID) {
  if (this.stream_compressor)
    return this.stream_compressor;

  this.stream_compressor = new ZLib(false);

  return this.stream_compressor;
};

Lastly, I ensure that I am using this non-dictionary zlib context stream and that I am generating a compressed data frame:

  var dframe = createDataFrame(this.getStreamCompressor(), {
    streamID: this.streamID,
    flags: fin ? enums.DATA_FLAG_FIN : enums.DATA_FLAG_COMPRESSED,
  }, Buffer.isBuffer(data) ? data : new Buffer(data, encoding));

Hopefully that will be sufficient. I had convinced myself that setting the DATA_FLAG_COMPRESSED would trigger compression of the data based on:

exports.createDataFrame = function(zlib, headers, data) {
  if (headers.flags & enums.DATA_FLAG_COMPRESSED) {
    data = zlib.deflate(data);
  }

//...
};

But, checking out the dialog in Chrome's about:net-internals // SPDY tab, it seems that the compression is not being set:

t=1307763599828 [st=113]     SPDY_SESSION_RECV_DATA  
                             --> flags = 0
                             --> size = 8577
                             --> stream_id = 7

(the flags are empty)

Bummer. Before investigating, I check the packets in Wireshark just to be sure that the flags are, in fact, empty:

               +----------------------------------+
 00 00 00 05   |C|       Stream-ID (31bits)       |
               +----------------------------------+
 02 00 00 1b   | Flags (8)  |  Length (24 bits)   |
               +----------------------------------+
 78 9c ca 48   |               Data               |
 cd c9 c9 57   +----------------------------------+
 48 2b ca cf
 55 28 4e 2d 2a 4b 2d 52 04 08 00 00 00 ff ff

So the compression is being applied after all (a flag of 0x02 means DATA_COMPRESSION ins SPDY). So, uh... Yay!

I think I will continue with data compression a bit longer tomorrow. I ought to take some time to verify that the data really is compressed and see if I can measure and time / size difference between compressed and non-compressed data streams.


Day #46