Hydra

‹prev | My Chain | next›

For quite some time I have wanted to mess about with hydra, the distributed testing framework from Nick Gauthier. His presentation to B'more on Rails a while back was nothing short of jaw dropping. At the time hydra lacked RSpec support (the best Ruby testing framework). Nick recently added RSpec support, so no more excuses...

First up, I gem install hydra (which pulls down version 0.16.2).

Then I run the specs from my current EEE Cooks site (done for last year's chain):

cstrom@whitefall:~/repos/eee-code$ time rake
(in /home/cstrom/repos/eee-code)   

==
Sinatra app spec
.......................................................

Finished in 17.72 seconds

55 examples, 0 failures

==
Helper specs
..........................................................................................

Finished in 0.44 seconds

90 examples, 0 failures

==
View specs
..............................................................................................................

Finished in 5.12 seconds

110 examples, 0 failures

real    0m34.556s
user    0m20.101s
sys     0m2.284s

Hmm... Not much of an opportunity to improve my spec time here (and this on my netbook). There is a reason that I use CouchDB, Sinatra and mock and stub heavily. No matter, let's see what hydra can do for me.

Per the excellent documentation, I add the following to my Rakefile:

# require the hydra codebase
require 'hydra'
# require the hydra rake task helpers
require 'hydra/tasks'

Since I am doing RSpec, I need to add a RSpec/hyrda task. To do that, I add this to my Rakefile:

# set up a new hydra testing task named 'hydra:spec' run with "rake hydra:spec"
Hydra::TestTask.new('hydra:spec') do |t|
  # add all files in the spec directory that end with "_spec.rb"
  t.add_files 'spec/**/*_spec.rb'
end

Now I have a hydra rake task:

cstrom@whitefall:~/repos/eee-code$ rake -T hydra
(in /home/cstrom/repos/eee-code)
rake hydra:spec  # Hydra Tests for hydra:spec

Let's give it a try:

cstrom@whitefall:~/repos/eee-code$ time rake hydra:spec
(in /home/cstrom/repos/eee-code)
Hydra Testing [##############################>] 14/14

real    0m24.885s
user    0m1.040s
sys     0m0.180s

Hmm... not too shabby. Just by running with hydra, the total time is 72% of the time when running without hydra. But is it really running my examples? I do have 14 spec files so it seems like it, but I have trust issues. What happens if I intentionally break an example? I change a sanity check example to expect failure rather than the success that it had expected, run the tests again and...

cstrom@whitefall:~/repos/eee-code$ time rake hydra:spec
(in /home/cstrom/repos/eee-code)
F......................................................

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:
/home/cstrom/.rvm/gems/ruby-1.8.7-p249/gems/hydra-0.16.2/lib/hydra/runner.rb:131:in `run_rspec_file'
/home/cstrom/.rvm/gems/ruby-1.8.7-p249/gems/hydra-0.16.2/lib/hydra/runner.rb:39:in `run_file'
/home/cstrom/.rvm/gems/ruby-1.8.7-p249/gems/hydra-0.16.2/lib/hydra/message/worker_messages.rb:19:in `handle'
... truncating a very large stacktrace ...
/home/cstrom/.rvm/gems/ruby-1.8.7-p249@global/bin/rake:19:in `load'
/home/cstrom/.rvm/gems/ruby-1.8.7-p249@global/bin/rake:19:
...............................

Ah cool! Good to know that my examples really are being run. There is a problem though in that I see this same failure 14 more times:

cstrom@whitefall:~/repos/eee-code$ time rake hydra:spec
(in /home/cstrom/repos/eee-code)
F......................................................

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

...............................

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

...............

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

................

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

..........................................................................................

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

...............

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

.......

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

.......

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

.......

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

......

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

...

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

..

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

..............

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

.

1)
'eee a CouchDB meal GET / should respond OK' FAILED
expected ok? to return false, got true
./spec/eee_spec.rb:48:

Hydra Testing [##############################>] 14/14

real    0m26.166s
user    0m1.116s
sys     0m0.140s

Ew. Well, not ideal, but I can live with that for now.

The real speed up that I ought to be able to get is by running on my remote machine persephone, which has an i7 with 4 cores. Hyperthreading actually gives me 8 process on that machine, which I define in my hydra.yml as:

workers:
  - type: ssh
    connect: cstrom@persephone.local
    directory: /home/cstrom/repos/eee-code
    runners: 8

Now, when I run my specs from my netbook, I get:

cstrom@whitefall:~/repos/eee-code$ time rake hydra:spec
(in /home/cstrom/repos/eee-code)
Hydra Testing [##############################>] 14/14

real    0m6.121s
user    0m1.076s
sys     0m0.152s

Now that is some speed up. It is 17% of the original speed. I am not realizing all of the speed gains that I can. Tomorrow, I will try this out with Cucumber, which currently takes just under three minutes to run.

Day #76