Wednesday, June 22, 2011

express-spdy: From the Ground Up

‹prev | My Chain | next›

Now that I have my first release of express-spdy out the door, it is time to circle back and deliver on some of the TODOs in the README. First up, is better documentation. Specifically, I am going to attempt to install express-spdy on a brand new system.

I will install onto a 64-bit Debian system. I am going with 64-bit because bitter experience has taught me that 64-bit is slightly more problematic than 32 bit. These instructions should work just fine for a 32 bit system as well. I am using Debian instead of Ubuntu to cut down slightly on the size of the VirtualBox image that I will use. Since Debian and Ubuntu share the same repository and since I will do this all at the command line, Ubuntu really offers no benefit in this case.

As my installer, I use debian-6.0.1a-amd64-i386-netinst.iso. For the install, I choose all defaults except the Software Packages at the end of the install—I choose none.

The only package that I install on my Debian box before getting started is ssh (for ease of copy & paste).

First up, I install Debian's build-essential. I do this out of habit on any new system, but it pulls in g++ for C++ code:
cstrom@debian:~$ sudo apt-get install build-essential
Next it is time to install openssl. This has to be the latest trunk to pull in the new Next Protocol Negotiation (NPN) stuff needed for SPDY. Since I am installing from a tarball and not a source code repository, I install in $HOME/src:
cstrom@debian:~$ mkdir src
cstrom@debian:~$ cd !$
cd src
cstrom@debian:~/src$ wget ftp://ftp.openssl.org/snapshot/openssl-SNAP-20110622.tar.gz
--2011-06-22 20:43:19-- ftp://ftp.openssl.org/snapshot/openssl-SNAP-20110622.tar.gz
Note: The openssl snapshots are only available for a rolling window of 4 days. In the future, I will check out ftp://ftp.openssl.org/snapshot/ for the most recent snapshot. This is a slight inconvenience, but beats installing CVS.

Now I can configure openssl:
./Configure shared --prefix=$HOME/local no-idea no-mdc2 no-rc5 zlib  enable-tlsext linux-elf
make depend
make
make install
And make depend. Unfortunately, I run into:
...
making depend in crypto/comp...
make[2]: Entering directory `/home/cstrom/src/openssl-SNAP-20110622/crypto/comp'
../../util/domd ../.. -MD gcc -- -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -I.. -I../.. -I../modes -I../asn1 -I../evp -I../../include -DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP224_64_GCC_128 -DOPENSSL_NO_GMP -DOPENSSL_NO_IDEA -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_MDC2 -DOPENSSL_NO_RC5 -DOPENSSL_NO_RFC3779 -DOPENSSL_NO_STORE -- comp_lib.c comp_err.c c_rle.c c_zlib.c
c_zlib.c:25:18: error: zlib.h: No such file or directory
make[2]: *** [depend] Error 1
make[2]: Leaving directory `/home/cstrom/src/openssl-SNAP-20110622/crypto/comp'
make[1]: *** [depend] Error 1
make[1]: Leaving directory `/home/cstrom/src/openssl-SNAP-20110622/crypto'
make: *** [depend] Error 1
After searching through Debian packages, I determine that I need to install lib64z1. But:
cstrom@debian:~/src/openssl-SNAP-20110622$ sudo apt-get install lib64z1
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package lib64z1
Er... What the hell? It most certainly is. Grr...

Will the 32-bit version install and work?
cstrom@debian:~/src/openssl-SNAP-20110622$ sudo apt-get install lib32z1-dev
That actually seems to do the trick as make depend completes successfully now. make depend completes, but make does not. Ugh.

After a bit of googling, I find that maybe I need:
cstrom@debian:~/src/openssl-SNAP-20110622$ sudo apt-get install zlib1g-dev
But I still end up with the same failures in make:
...
gcc -c -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -c -o x86cpuid.o x86cpuid.s
x86cpuid.s: Assembler messages:
x86cpuid.s:8: Error: suffix or operands invalid for `push'
x86cpuid.s:9: Error: suffix or operands invalid for `push'
x86cpuid.s:10: Error: suffix or operands invalid for `push'
x86cpuid.s:11: Error: suffix or operands invalid for `push'
x86cpuid.s:13: Error: suffix or operands invalid for `pushf'
x86cpuid.s:14: Error: suffix or operands invalid for `pop'
x86cpuid.s:17: Error: suffix or operands invalid for `push'
x86cpuid.s:18: Error: suffix or operands invalid for `popf'
x86cpuid.s:19: Error: suffix or operands invalid for `pushf'
x86cpuid.s:20: Error: suffix or operands invalid for `pop'
...
Ugh. It seems that my Configure line was erroneous. I am working on 64-bit here, but am targeting vanilla linux-elf. So I try again:
cstrom@debian:~/src/openssl-SNAP-20110622$ ./Configure shared --prefix=$HOME/local no-idea no-mdc2 no-rc5 zlib  enable-tlsext linux-x86_64
With that, I am ready to install node. I need node 0.5.0-pre, which means installing from the git repository. That means git:
cstrom@debian:~/repos$ sudo apt-get install git-core
Since this is coming from a source code repository, I put the copy into $HOME/repos:
cstrom@debian:~$ mkdir repos
cstrom@debian:~$ cd !$
cd repos
And clone the node repository:
cstrom@debian:~/repos$ git clone https://github.com/joyent/node.git
Now I can configure node to use my freshly installed edge-openssl:
cstrom@debian:~/repos/node$ ./configure --openssl-includes=$HOME/local/include --openssl-libpath=$HOME/local/lib --prefix=$HOME/local/node-v0.5.0-pre
/usr/bin/env: python: No such file or directory
But first, I need to install python:
cstrom@debian:~/repos/node$ sudo apt-get install python
That does the trick. After that, I can:
./configure --openssl-includes=$HOME/local/include --openssl-libpath=$HOME/local/lib --prefix=$HOME/local/node-v0.5.0-pre
make
make install
To easily use my locally installed openssl and node binaries, shared objects, etc., I add the following to my $HOME/.bashrc:
# For locally installed binaries
export LD_LIBRARY_PATH=$HOME/local/lib
PATH=$HOME/local/bin:$PATH
PKG_CONFIG_PATH=$HOME/local/lib/pkgconfig
CPATH=$HOME/local/include
export MANPATH=$HOME/local/share/man:/usr/share/man

# For node.js work. For more info, see:
# http://blog.nodejs.org/2011/04/04/development-environment/
for i in $HOME/local/*; do
[ -d $i/bin ] && PATH="${i}/bin:${PATH}"
[ -d $i/sbin ] && PATH="${i}/sbin:${PATH}"
[ -d $i/include ] && CPATH="${i}/include:${CPATH}"
[ -d $i/lib ] && LD_LIBRARY_PATH="${i}/lib:${LD_LIBRARY_PATH}"
[ -d $i/lib/pkgconfig ] && PKG_CONFIG_PATH="${i}/lib/pkgconfig:${PKG_CONFIG_PATH}"
[ -d $i/share/man ] && MANPATH="${i}/share/man:${MANPATH}"
done
I log out and log back in to ensure that my .bashrc changes are picked up (I could also have manually sourced ~/.bashrc). Now it is time to install the node package manager (NPM):
cstrom@debian:~$ which node
/home/cstrom/local/node-v0.5.0-pre/bin/node
cstrom@debian:~$ curl http://npmjs.org/install.sh | sh
-bash: curl: command not found
But first I need to install curl:
cstrom@debian:~$ sudo apt-get install curl
Now, my npm installs successfully:
cstrom@debian:~$ curl http://npmjs.org/install.sh | sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3902 100 3902 0 0 12247 0 --:--:-- --:--:-- --:--:-- 32247
fetching: http://registry.npmjs.org/npm/-/npm-1.0.14.tgz
0.5.0-pre
1.0.14
prefix=/home/cstrom/local/node-v0.5.0-pre

This script will find and eliminate any shims, symbolic
links, and other cruft that was installed by npm 0.x.

Is this OK? enter 'yes' or 'no'
yes

All clean!
! [ -d .git ] || git submodule update --init
node cli.js cache clean
node cli.js rm npm -g -f --loglevel error
node cli.js install -g -f
/home/cstrom/local/node-v0.5.0-pre/bin/npm -> /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/npm/bin/npm.js
/home/cstrom/local/node-v0.5.0-pre/bin/npm_g -> /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/npm/bin/npm.js
/home/cstrom/local/node-v0.5.0-pre/bin/npm-g -> /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/npm/bin/npm.js
npm@1.0.14 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/npm
It worked
I am close now.

Next I install express.js express.js "globally". I do this so that I can run the express executable generator that comes with express:
cstrom@debian:~$ npm install -g express
/home/cstrom/local/node-v0.5.0-pre/bin/express -> /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express/bin/express
mime@1.2.2 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express/node_modules/mime
qs@0.1.0 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express/node_modules/qs
connect@1.5.1 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express/node_modules/connect
express@2.3.12 /home/cstrom/local/node-v0.5.0-pre/lib/node_modules/express
Since I am likely to put my SPDY-ized express.js application under source code control, I create my express.js app in $HOME/repos:
cstrom@debian:~$ cd repos/
cstrom@debian:~/repos$ express test
create : test
create : test/package.json
create : test/app.js
create : test/public/javascripts
create : test/logs
create : test/pids
create : test/public/images
create : test/views
create : test/views/layout.jade
create : test/views/index.jade
create : test/public/stylesheets
create : test/public/stylesheets/style.css
In my test app, I install the new express-spdy package from npm:
cstrom@debian:~/repos$ cd test
cstrom@debian:~/repos/test$ npm install express-spdy

> zlibcontext@1.0.7 install /home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext
> ./configure && make

Setting srcdir to : /home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext
Setting blddir to : /home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext/build
Checking for program g++ or c++ : /usr/bin/g++
Checking for program cpp : /usr/bin/cpp
Checking for program ar : /usr/bin/ar
Checking for program ranlib : /usr/bin/ranlib
Checking for g++ : ok
Checking for node path : not found
Checking for node prefix : ok /home/cstrom/local/node-v0.5.0-pre
Checking for library z : yes
'configure' finished successfully (0.431s)
node-waf build
Waf: Entering directory `/home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext/build'
[1/2] cxx: src/node_zlib.cc -> build/default/src/node_zlib_1.o
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:568: warning: ‘int ev_is_default_loop()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:804: warning: ‘void ev_loop(int)’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:805: warning: ‘void ev_unloop(int)’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:806: warning: ‘void ev_default_destroy()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:807: warning: ‘void ev_default_fork()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:809: warning: ‘unsigned int ev_loop_count()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:810: warning: ‘unsigned int ev_loop_depth()’ defined but not used
/home/cstrom/local/node-v0.5.0-pre/include/node/ev/ev.h:811: warning: ‘void ev_loop_verify()’ defined but not used
[2/2] cxx_link: build/default/src/node_zlib_1.o -> build/default/zlib_bindings.node
Waf: Leaving directory `/home/cstrom/repos/test/node_modules/express-spdy/node_modules/spdy/node_modules/zlibcontext/build'
'build' finished successfully (0.973s)
express-spdy@0.0.1 ./node_modules/express-spdy
├── express@2.3.12 (mime@1.2.2 qs@0.1.0 connect@1.5.1)
├── connect-spdy@0.0.1 (connect@1.5.1)
└── spdy@0.0.1
SPDY requires SSL, which requires keys. I copy them from the node-spdy package, which was installed as a dependency of express-spdy:
cstrom@debian:~/repos/test$ mkdir keys
cstrom@debian:~/repos/test$ cp node_modules/express-spdy/node_modules/spdy/keys/spdy* keys/
Lastly (I hope), I modify the generated app.js to use those keys and to use express-spdy instead of vanilla express.js:
var express = require('express-spdy')
, fs = require('fs');

var app = module.exports = express.createServer({
key: fs.readFileSync(__dirname + '/keys/spdy-key.pem'),
cert: fs.readFileSync(__dirname + '/keys/spdy-cert.pem'),
ca: fs.readFileSync(__dirname + '/keys/spdy-csr.pem'),
NPNProtocols: ['spdy/2']
});

//var express = require('express');

//var app = module.exports = express.createServer();
After starting the app up, however, I am greeted with:
cstrom@debian:~/repos/test$ node app.js
Express server listening on port 3000 in development mode
Error: Cannot find module 'jade'
...
Arrrgh. I never remember to install the jade templating engine. Even though the express.js generated app includes jade templates. Ah well, that is easily enough resolved:
cstrom@debian:~/repos/test$ npm install jade
jade@0.12.3 ./node_modules/jade
With that, the apps starts just fine:
cstrom@debian:~/repos/test$ node app.js
Express server listening on port 3000 in development mode
More importantly, I get a valid express site when I access the site with Chrome:



Most importantly, I get an actual legitimate SPDY session as evidenced by the SPDY tab in about:net-internals:



Yay! That ought to sufficient to write up some express-spdy install instructions.


Day #56

No comments:

Post a Comment