Hydra, Cucumber, and CouchDB

‹prev | My Chain | next›

Last night I got some pretty decent speed up in my RSpec examples thanks to hydra. But really, I did not need much help—the couple hundred specs all run in about 30 seconds even on my little netbook.

What I need help with are my Cucumber scenarios. On my netbook (whitefall):

cstrom@whitefall:~/repos/eee-code$ time cucumber
...

39 scenarios (1 pending, 38 passed)
344 steps (1 pending, 343 passed)
2m30.583s

real    2m46.764s
user    0m26.242s
sys     0m2.876s

That's just crazy.

The story is not quite as bad on my 16-inch i7 (persephone):

cstrom@persephone:~/repos/eee-code$ time cucumber
...

39 scenarios (1 pending, 38 passed)
344 steps (1 pending, 343 passed)
0m38.159s

real    0m40.900s
user    0m8.520s
sys     0m0.940s

But who the hell wants to lug about a huge-ass computer? I really do fancy coding on my netbook. Let's see if hydra can help.

I add the hydra/Cucumber rake tasks to my Rakefile:

# require the hydra codebase
require 'hydra'
# require the hydra rake task helpers
require 'hydra/tasks'

# set up a new hydra testing task named 'hydra:spec' run with "rake hydra:spec"
Hydra::TestTask.new('hydra:spec') do |t|
  # add all files in the spec directory that end with "_spec.rb"
  t.add_files 'spec/**/*_spec.rb'
end

# set up a new hydra testing task named 'hydra:cucumber' run with "rake hydra:cucumber"
Hydra::TestTask.new('hydra:cucumber') do |t|
  # add all files in the features directory that end with ".feature"
  t.add_files 'features/**/*.feature'
end

When I run that task from my netbook, however, I find:

cstrom@whitefall:~/repos/eee-code$ rake hydra:cucumber
(in /home/cstrom/repos/eee-code)
(::) failed steps (::)                                      

Resource not found (RestClient::ResourceNotFound)
/usr/lib/ruby/1.8/net/http.rb:543:in `start'
./features/support/../../eee.rb:179:in `GET (?-mix:\/recipes\/(\d+)\/(\d+)\/(\d+)\/?(.*))'
(eval):2:in `visit'
./features/step_definitions/draft.rb:26:in `/^I show "Recipe #1"$/'
features/draft_recipes.feature:12:in `When I show "Recipe #1"'


(::) failed steps (::)                                      

Resource not found (RestClient::ResourceNotFound)
/usr/lib/ruby/1.8/net/http.rb:543:in `start'
./features/support/../../eee.rb:179:in `GET (?-mix:\/recipes\/(\d+)\/(\d+)\/(\d+)\/?(.*))'
(eval):2:in `visit'
./features/step_definitions/draft.rb:26:in `/^I show "Recipe #1"$/'
features/draft_recipes.feature:12:in `When I show "Recipe #1"'


Hydra Testing [##############################>] 10/10

Hunh? I just ran those scenarios on two different machines and they passed with flying colors. What gives?

This, I think can be traced back to how I handle my CouchDB "transaction" rollbacks—I don't. More to the point, I drop the test database and recreate it after each scenario using Cucumber Before/After blocks:

Before do
  # For mocking & stubbing in Cucumber
  $rspec_mocks ||= Spec::Mocks::Space.new

  # Create the DB
  RestClient.put @@db, { }

  # Upload the design documents with a super-easy gem :)
  CouchDocs.upload_dir(@@db, 'couch')
end

After do
  begin
    $rspec_mocks.verify_all
  ensure
    $rspec_mocks.reset_all
  end

  # Delete the DB
  RestClient.delete @@db
  sleep 0.5
end

Dropping and recreating databases might seem like a ton of overhead, but that is only because you are still stuck in a traditional relational database world. I live in the happy world of CouchDB where dropping/creating databases is microseconds. This approach has served me well, until now.

What is happening here is that one of my 8 hydra workers is finishing a CouchDB scenario and dropping the testing database at the same time that another process is trying to do some work in that same test DB. Since the DB is (momentarily) not there, I get RestClient not found failures.

To work around this, I create a different database for each scenario using Time#usec:

Before do
  # For mocking & stubbing in Cucumber
  $rspec_mocks ||= Spec::Mocks::Space.new

  # Create the DB
  @@db = "http://localhost:5984/eee-test-#{Time.now.usec}"
  RestClient.put @@db, { }

  # Upload the design documents with a super-easy gem :)
  CouchDocs.upload_dir(@@db, 'couch')
end

(I had tried rand, but it was not quite random enough).

With that change in place, I run my hydra/cucumbers again... only to still receive a similar kind of failure. But of course! I made the change on my netbook, but not on my i7. Ugh. Do I really need to copy this change over? Every time I make a change, do I need to copy it over by hand? That would stink as a normal development workflow.

Of course I do not have to do anything of the sort. Hyrda has a syncing mechanism that does just this. To set this up, I add a sync section to my hydra.yml configuration:

workers:
  - type: ssh
    connect: cstrom@persephone.local
    directory: /home/cstrom/repos/eee-code
    runners: 8
sync:
    directory: /home/cstrom/repos/eee-code
    exclude:
      - tmp
      - log
      - doc

With that, I can run my hydra specs from my little netbook one more time:



All specs pass, but Hydra reports them as red because of the one pending example. That aside, I can now run my rather large Cucumber suite from the comfort of my netbook in less than half the time it took to run on my i7. That is a big win.

Day #77