Implementing Recipe Search, Part 1

‹prev | My Chain | next›

With recipe show / details done, next up is recipe search. A pair of spikes helped me to understand how I might implement full text searching. Now it is time to do it for real.

The feature, as described in Cucumber format:

Feature: Search for recipes

  So that I can find one recipe among many
  As a web user
  I want to be able search recipes

The first scenario, also in cucumber format:

    Scenario: Matching a word in the ingredient list in full recipe search

      Given a "pancake" recipe with "chocolate chips" in it
      And a "french toast" recipe with "eggs" in it
      When I search for "chocolate"
      Then I should see the "pancake" recipe in the search results
      And I should not see the "french toast" recipe in the search results

It is a good thing I finally figured out full document indexing the other day, otherwise I might have to defer this particular scenario to another day.

The first to Given steps are easy enough to implement at this point—several similar ones were needed during the recipe details feature work. By way of illustration, the a "french toast" recipe with "eggs" in it step is implemented in the newly created features/step_definitions/recipe_search.rb as:

Given /^a "french toast" recipe with "eggs" in it$/ do
  @date = Date.new(2009, 4, 12)
  @title = "French Toast"
  @permalink = @date.to_s + "-" + @title.downcase.gsub(/\W/, '-')

  recipe = {
    :title => @title,
    :date  => @date,
    :preparations => [
      {
        'quantity' => '1',
        'ingredient' => { 'name' => 'egg'}
      }
    ]
  }

  RestClient.put "#{@@db}/#{@permalink}",
    recipe.to_json,
    :content_type => 'application/json'
end

Next up is the When I search for "chocolate" step. I need to move into the application in order to implement search. Before moving into the nitty-gritty of the application, I realize that my test DB does not have the full document, full text search definition that I added to my development DB the other night.

Uh-oh.

I am adding something to my test DB that I already added to my development DB. That same something will need to be in my production DB. Sounds like database migrations to me. Ugh. Something for another day. For now I will add it to the the Before block of features/support/env.rb, but will have to address this in the very near future. The Before block with the new code:

Before do
  RestClient.put @@db, { }

  # TODO need to accomplish this via CouchDB migrations
  lucene_index_function = <<_JS
function(doc) {
  var ret = new Document();

  function idx(obj) {
    for (var key in obj) {
      switch (typeof obj[key]) {
        case 'object':
          idx(obj[key]);
          break;
        case 'function':
          break;
        default:
          ret.field(key, obj[key]);
          ret.field('all', obj[key]);
          break;
      }
    }
  }

  idx(doc);

  return ret;
}
_JS

  doc = { 'transform' => lucene_index_function }

  RestClient.put "#{@@db}/_design/lucene",
    doc.to_json,
    :content_type => 'application/json'
end

To try this out, I need to implement my When search step. In order to do that, I need to work my way into the code so that I can implement the search.

For the target scenario, the user should be able to search for a term anywhere (title, summary, ingredients) in the recipe document. This is the "all" field that was created the other night. The API that I would like to expose is that if I query for "eggs" in the Sinatra app, it should be passed on as an "all" search to couchdb. The example that describes this behavior:

  describe "GET /recipes/search" do
    it "should retrieve search results from couchdb-lucene" do
      RestClient.should_receive(:get).
        with("#{@@db}/_fti?q=all:eggs").
        and_return('{"total_rows":1}')

      get "/recipes/search?q=eggs"
    end
  end

The code that implements this is:

get '/recipes/search' do
  data = RestClient.get "#{@@db}/_fti?q=all:#{params[:q]}"
  @results = JSON.parse(data)

  ["results:", @results['total_rows'].to_s]
end

This example does not render any results. The current step that is being implemented is the "When I search...". This example and solution are the simplest things that work. I will worry about the results when I get to the Then steps.

After working my way out to implement the "When I search" step with a simple Webrat visit, I am ready to give the Then steps a try. Without getting into too many details with regards to the output format, I try to implement this example:

    it "should include a link to a match" do
      RestClient.should_receive(:get).
        with("#{@@db}/_fti?q=all:eggs").
        and_return('{"total_rows":1,"rows":[{"_id":"007"}]}')

      get "/recipes/search?q=eggs"
      response.should have_selector("a", :href => "/recipes/007")
    end

With this code:

get '/recipes/search' do
  data = RestClient.get "#{@@db}/_fti?q=all:#{params[:q]}"
  @results = JSON.parse(data)

  ["results: #{@results['total_rows']}<br/>"] +
    @results['rows'].map do |result|
      %Q|<a href="/recipes/#{result['_id']}">title</a>|
    end
end

The search results iterate over the "rows" in the results, mapping to the desired recipe link. Without even bothering to get the title included in the output, I pop back out to the cucumber scenario to see if this attempt at a Then step might succeed:

Then /^I should see the "pancake" recipe in the search results$/ do
  response.should have_selector("a", :href => "/recipes/#{@pancake_permalink}")
end

Unfortunately, it does not:

cstrom@jaynestown:~/repos/eee-code$ cucumber features/recipe_search.feature -n \
    -s "Matching a word in the ingredient list in full recipe search"
Feature: Search for recipes

  So that I can find one recipe among many
  As a web user
  I want to be able search recipes
  Scenario: Matching a word in the ingredient list in full recipe search
    Given a "pancake" recipe with "chocolate chips" in it
    And a "french toast" recipe with "eggs" in it
    When I search for "chocolate"
    Then I should see the "pancake" recipe in the search results
      expected following output to contain a <a href='/recipes/2009-04-12-buttermilk-chocolate-chip-pancakes'/> tag:
      <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
      <html><body><p>results: 0<br></p></body></html> (Spec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/recipe_search.rb:51:in `Then /^I should see the "pancake" recipe in the search results$/'
      features/recipe_search.feature:12:in `Then I should see the "pancake" recipe in the search results'
    And I should not see the "french toast" recipe in the search results


1 scenario
3 steps passed
1 step failed
1 step pending (1 with no step definition)

You can use these snippets to implement pending steps which have no step definition:

Then /^I should not see the "french toast" recipe in the search results$/ do
end

Hmm. I am not getting any search results back. My best guess is that my lucene design document is not working as expected when uploading via RestClient.

It is late, so I will have to test that guess tomorrow. I do not mind stopping at Red during the Red-Green-Refactor cycle—I know exactly where to pick up tomorrow.
(commit)