express-spdy: From the Ground Up

‹prev | My Chain | next›

Now that I have my first release of express-spdy out the door, it is time to circle back and deliver on some of the TODOs in the README. First up, is better documentation. Specifically, I am going to attempt to install express-spdy on a brand new system.

I will install onto a 64-bit Debian system. I am going with 64-bit because bitter experience has taught me that 64-bit is slightly more problematic than 32 bit. These instructions should work just fine for a 32 bit system as well. I am using Debian instead of Ubuntu to cut down slightly on the size of the VirtualBox image that I will use. Since Debian and Ubuntu share the same repository and since I will do this all at the command line, Ubuntu really offers no benefit in this case.

As my installer, I use debian-6.0.1a-amd64-i386-netinst.iso. For the install, I choose all defaults except the Software Packages at the end of the install—I choose none.

The only package that I install on my Debian box before getting started is ssh (for ease of copy & paste).

First up, I install Debian's build-essential. I do this out of habit on any new system, but it pulls in g++ for C++ code:

cstrom@debian:~$ sudo apt-get install build-essential

Next it is time to install openssl. This has to be the latest trunk to pull in the new Next Protocol Negotiation (NPN) stuff needed for SPDY. Since I am installing from a tarball and not a source code repository, I install in $HOME/src:

cstrom@debian:~$ mkdir src
cstrom@debian:~$ cd !$
cd src
cstrom@debian:~/src$ wget ftp://ftp.openssl.org/snapshot/openssl-SNAP-20110622.tar.gz
--2011-06-22 20:43:19--  ftp://ftp.openssl.org/snapshot/openssl-SNAP-20110622.tar.gz

Note: The openssl snapshots are only available for a rolling window of 4 days. In the future, I will check out ftp://ftp.openssl.org/snapshot/ for the most recent snapshot. This is a slight inconvenience, but beats installing CVS.

Now I can configure openssl:

./Configure shared --prefix=$HOME/local no-idea no-mdc2 no-rc5 zlib  enable-tlsext linux-elf
make depend
make
make install

And make depend. Unfortunately, I run into:

...
making depend in crypto/comp...
make[2]: Entering directory `/home/cstrom/src/openssl-SNAP-20110622/crypto/comp'
../../util/domd ../.. -MD gcc -- -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -I.. -I../.. -I../modes -I../asn1 -I../evp -I../../include  -DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP224_64_GCC_128 -DOPENSSL_NO_GMP -DOPENSSL_NO_IDEA -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_MDC2 -DOPENSSL_NO_RC5 -DOPENSSL_NO_RFC3779 -DOPENSSL_NO_STORE -- comp_lib.c comp_err.c c_rle.c c_zlib.c
c_zlib.c:25:18: error: zlib.h: No such file or directory
make[2]: *** [depend] Error 1
make[2]: Leaving directory `/home/cstrom/src/openssl-SNAP-20110622/crypto/comp'
make[1]: *** [depend] Error 1
make[1]: Leaving directory `/home/cstrom/src/openssl-SNAP-20110622/crypto'
make: *** [depend] Error 1

After searching through Debian packages, I determine that I need to install lib64z1. But:

cstrom@debian:~/src/openssl-SNAP-20110622$ sudo apt-get install lib64z1
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package lib64z1

Er... What the hell? It most certainly is. Grr...

Will the 32-bit version install and work?

cstrom@debian:~/src/openssl-SNAP-20110622$ sudo apt-get install lib32z1-dev

That actually seems to do the trick as make depend completes successfully now. make depend completes, but make does not. Ugh.

After a bit of googling, I find that maybe I need:

cstrom@debian:~/src/openssl-SNAP-20110622$ sudo apt-get install zlib1g-dev

But I still end up with the same failures in make:

...
gcc -c -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -c  -o x86cpuid.o x86cpuid.s
x86cpuid.s: Assembler messages:
x86cpuid.s:8: Error: suffix or operands invalid for `push'
x86cpuid.s:9: Error: suffix or operands invalid for `push'
x86cpuid.s:10: Error: suffix or operands invalid for `push'
x86cpuid.s:11: Error: suffix or operands invalid for `push'
x86cpuid.s:13: Error: suffix or operands invalid for `pushf'
x86cpuid.s:14: Error: suffix or operands invalid for `pop'
x86cpuid.s:17: Error: suffix or operands invalid for `push'
x86cpuid.s:18: Error: suffix or operands invalid for `popf'
x86cpuid.s:19: Error: suffix or operands invalid for `pushf'
x86cpuid.s:20: Error: suffix or operands invalid for `pop'
...

Ugh. It seems that my Configure line was erroneous. I am working on 64-bit here, but am targeting vanilla linux-elf. So I try again:

cstrom@debian:~/src/openssl-SNAP-20110622$ ./Configure shared --prefix=$HOME/local no-idea no-mdc2 no-rc5 zlib  enable-tlsext linux-x86_64

With that, I am ready to install node. I need node 0.5.0-pre, which means installing from the git repository. That means git:

cstrom@debian:~/repos$ sudo apt-get install git-core

Since this is coming from a source code repository, I put the copy into $HOME/repos:

cstrom@debian:~$ mkdir repos
cstrom@debian:~$ cd !$
cd repos

And clone the node repository:

cstrom@debian:~/repos$ git clone https://github.com/joyent/node.git

Now I can configure node to use my freshly installed edge-openssl:

cstrom@debian:~/repos/node$ ./configure --openssl-includes=$HOME/local/include --openssl-libpath=$HOME/local/lib --prefix=$HOME/local/node-v0.5.0-pre
/usr/bin/env: python: No such file or directory

But first, I need to install python:

cstrom@debian:~/repos/node$ sudo apt-get install python

That does the trick. After that, I can:

./configure --openssl-includes=$HOME/local/include --openssl-libpath=$HOME/local/lib --prefix=$HOME/local/node-v0.5.0-pre
make
make install

To easily use my locally installed openssl and node binaries, shared objects, etc., I add the following to my $HOME/.bashrc:

# For locally installed binaries
export LD_LIBRARY_PATH=$HOME/local/lib
PATH=$HOME/local/bin:$PATH
PKG_CONFIG_PATH=$HOME/local/lib/pkgconfig
CPATH=$HOME/local/include
export MANPATH=$HOME/local/share/man:/usr/share/man

# For node.js work. For more info, see:
# http://blog.nodejs.org/2011/04/04/development-environment/
for i in $HOME/local/*; do
  [ -d $i/bin ] && PATH="${i}/bin:${PATH}"
  [ -d $i/sbin ] && PATH="${i}/sbin:${PATH}"
  [ -d $i/include ] && CPATH="${i}/include:${CPATH}"
  [ -d $i/lib ] && LD_LIBRARY_PATH="${i}/lib:${LD_LIBRARY_PATH}"
  [ -d $i/lib/pkgconfig ] && PKG_CONFIG_PATH="${i}/lib/pkgconfig:${PKG_CONFIG_PATH}"
  [ -d $i/share/man ] && MANPATH="${i}/share/man:${MANPATH}"
done

I log out and log back in to ensure that my .bashrc changes are picked up (I could also have manually sourced ~/.bashrc). Now it is time to install the node package manager (NPM):

cstrom@debian:~$ which node
/home/cstrom/local/node-v0.5.0-pre/bin/node
cstrom@debian:~$ curl http://npmjs.org/install.sh | sh
-bash: curl: command not found

But first I need to install curl:

cstrom@debian:~$ sudo apt-get install curl

Now, my npm installs successfully:

cstrom@debian:~$ curl http://npmjs.org/install.sh | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3902  100  3902    0     0  12247      0 --:--:-- --:--:-- --:--:-- 32247
fetching: http://registry.npmjs.org/npm/-/npm-1.0.14.tgz
0.5.0-pre
1.0.14
prefix=/home/cstrom/local/node-v0.5.0-pre

This script will find and eliminate any shims, symbolic
links, and other cruft that was installed by npm 0.x.

Is this OK? enter 'yes' or 'no'
yes

All clean!
! [ -d .git ] || git submodule update --init
node cli.js cache clean
node cli.js rm npm -g -f --loglevel error
node cli.js install -g -f
/home/cstrom/local/node-v0.5.0-pre/bin/npm -> /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/npm/bin/npm.js
/home/cstrom/local/node-v0.5.0-pre/bin/npm_g -> /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/npm/bin/npm.js
/home/cstrom/local/node-v0.5.0-pre/bin/npm-g -> /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/npm/bin/npm.js
npm@1.0.14 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/npm
It worked

I am close now.

Next I install express.js express.js "globally". I do this so that I can run the express executable generator that comes with express:

cstrom@debian:~$ npm install -g express
/home/cstrom/local/node-v0.5.0-pre/bin/express -> /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express/bin/express
mime@1.2.2 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express/node_modules/mime 
qs@0.1.0 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express/node_modules/qs 
connect@1.5.1 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express/node_modules/connect 
express@2.3.12 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express 

Since I am likely to put my SPDY-ized express.js application under source code control, I create my express.js app in $HOME/repos:

cstrom@debian:~$ cd repos/
cstrom@debian:~/repos$ express test
   create : test
   create : test/package.json
   create : test/app.js
   create : test/public/javascripts
   create : test/logs
   create : test/pids
   create : test/public/images
   create : test/views
   create : test/views/layout.jade
   create : test/views/index.jade
   create : test/public/stylesheets
   create : test/public/stylesheets/style.css

In my test app, I install the new express-spdy package from npm:

cstrom@debian:~/repos$ cd test
cstrom@debian:~/repos/test$ npm install express-spdy

> zlibcontext@1.0.7 install /home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext
> ./configure && make

Setting srcdir to                        : /home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext 
Setting blddir to                        : /home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext/build 
Checking for program g++ or c++          : /usr/bin/g++ 
Checking for program cpp                 : /usr/bin/cpp 
Checking for program ar                  : /usr/bin/ar 
Checking for program ranlib              : /usr/bin/ranlib 
Checking for g++                         : ok  
Checking for node path                   : not found 
Checking for node prefix                 : ok /home/cstrom/local/node-v0.5.0-pre 
Checking for library z                   : yes 
'configure' finished successfully (0.431s)
node-waf build
Waf: Entering directory `/home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext/build'
[1/2] cxx: src/node_zlib.cc -> build/default/src/node_zlib_1.o
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:568: warning: ‘int ev_is_default_loop()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:804: warning: ‘void ev_loop(int)’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:805: warning: ‘void ev_unloop(int)’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:806: warning: ‘void ev_default_destroy()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:807: warning: ‘void ev_default_fork()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:809: warning: ‘unsigned int ev_loop_count()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:810: warning: ‘unsigned int ev_loop_depth()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:811: warning: ‘void ev_loop_verify()’ defined but not used
[2/2] cxx_link: build/default/src/node_zlib_1.o -> build/default/zlib_bindings.node
Waf: Leaving directory `/home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext/build'
'build' finished successfully (0.973s)
express-spdy@0.0.1 ./node_modules/express-spdy 
├── express@2.3.12 (mime@1.2.2 qs@0.1.0 connect@1.5.1)
├── connect-spdy@0.0.1 (connect@1.5.1)
└── spdy@0.0.1

SPDY requires SSL, which requires keys. I copy them from the node-spdy package, which was installed as a dependency of express-spdy:

cstrom@debian:~/repos/test$ mkdir keys
cstrom@debian:~/repos/test$ cp node_modules/express-spdy/node_modules/spdy/keys/spdy* keys/

Lastly (I hope), I modify the generated app.js to use those keys and to use express-spdy instead of vanilla express.js:

var express = require('express-spdy')
  , fs = require('fs');

var app = module.exports = express.createServer({
  key: fs.readFileSync(__dirname + '/keys/spdy-key.pem'),
  cert: fs.readFileSync(__dirname + '/keys/spdy-cert.pem'),
  ca: fs.readFileSync(__dirname + '/keys/spdy-csr.pem'),
  NPNProtocols: ['spdy/2']
});

//var express = require('express');

//var app = module.exports = express.createServer();

After starting the app up, however, I am greeted with:

cstrom@debian:~/repos/test$ node app.js
Express server listening on port 3000 in development mode
Error: Cannot find module 'jade'
...

Arrrgh. I never remember to install the jade templating engine. Even though the express.js generated app includes jade templates. Ah well, that is easily enough resolved:

cstrom@debian:~/repos/test$ npm install jade
jade@0.12.3 ./node_modules/jade

With that, the apps starts just fine:

cstrom@debian:~/repos/test$ node app.js
Express server listening on port 3000 in development mode

More importantly, I get a valid express site when I access the site with Chrome:



Most importantly, I get an actual legitimate SPDY session as evidenced by the SPDY tab in about:net-internals:



Yay! That ought to sufficient to write up some express-spdy install instructions.


Day #56