Tuesday, April 21, 2009

Searching Deep Data Structures

‹prev | My Chain | next›

On tap for tonight, the "Searching Ingredients" scenario:
    Scenario: Searching ingredients

Given a "pancake" recipe with "chocolate chips" in it
And a "french toast" recipe with "eggs" in it and a summary of "does not go well with chocolate"
And a 0.5 second wait to allow the search index to be updated
When I search ingredients for "chocolate"
Then I should see the "pancake" recipe in the search results
And I should not see the "french toast" recipe in the search results
This is another fielded search like yesterday's Searching titles, so it ought to be easy, right?

Turns out not so much.

The cause of the trouble can be seen in the (new) step definition that implements Given a "french toast" recipe with "eggs" in it and a summary of "does not go well with chocolate":
Given /^a "(.+)" recipe with "(.+)" in it and a summary of "(.+)"$/ do |title, ingredient, summary|
date = Date.new(2009, 4, 21)
permalink = "id-#{title.gsub(/\W/, '-')}"

@pancake_recipe = {
:title => title,
:date => date,
:summary => summary,
:preparations => [
{
'ingredient' => {
'name' => ingredient
}
}
]
}

RestClient.put "#{@@db}/#{permalink}",
@pancake_recipe.to_json,
:content_type => 'application/json'
end
The ingredient ("eggs" as set in the scenario text) is buried inside a single "preparation". The current indexing algorithm would index the ingredient name not in the 'ingredient' field, but in the 'name' field (because it is indexed by key). Just to double-check, I implement the When I search ingredients for "chocolate" step as:
When /^I search ingredients for "(.+)"$/ do |keyword|
visit("/recipes/search?q=ingredient:#{keyword}")
end
Running cucumber fails, as expected:
cstrom@jaynestown:~/repos/eee-code$ cucumber features/recipe_search.feature -n \
-s "Searching ingredients"
Feature: Search for recipes

So that I can find one recipe among many
As a web user
I want to be able search recipes
Scenario: Searching ingredients
Given a "pancake" recipe with "chocolate chips" in it
And a "french toast" recipe with "eggs" in it and a summary of "does not go well with chocolate"
And a 0.5 second wait to allow the search index to be updated
When I search ingredients for "chocolate"
Then I should see the "pancake" recipe in the search results
expected following output to contain a <a href='/recipes/id-pancake'>pancake</a> tag:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html><body><table><tr>
<th>Name</th>
<th>Date</th>
</tr></table></body></html> (Spec::Expectations::ExpectationNotMetError)
./features/step_definitions/recipe_search.rb:127:in `Then /^I should see the "(.+)" recipe in the search results$/'
features/recipe_search.feature:49:in `Then I should see the "pancake" recipe in the search results'
And I should not see the "french toast" recipe in the search results


1 scenario
4 steps passed
1 step failed
1 step skipped
I do not want to search for "name:eggs", I want to search for "ingredient:eggs", so I have some work to do with the couchdb-lucene indexer. Fortunately, it is not too much work. All that is needed is a special case to handle ingredients:
          /* Handle ingredients as a special case */
if (key == 'preparations') {
var ingredients = [];
for (var i=0; i<obj[key].length; i++) {
ingredients.push(obj[key][i]['ingredient']['name']);
}
ret.field('ingredient', ingredients.join(', '), 'yes');
}
This block builds up the ingredients array with each of the ingredient names. It then takes that array, joins it with commas and indexes the result in the 'ingredient' field. For good measure it stores this value in the index for easy retrieval.

Stepping all the way back out to the cucumber test, everything now passes:
cstrom@jaynestown:~/repos/eee-code$ cucumber features/recipe_search.feature -n -s "Searching ingredients"
Feature: Search for recipes

So that I can find one recipe among many
As a web user
I want to be able search recipes
Scenario: Searching ingredients
Given a "pancake" recipe with "chocolate chips" in it
And a "french toast" recipe with "eggs" in it and a summary of "does not go well with chocolate"
And a 0.5 second wait to allow the search index to be updated
When I search ingredients for "chocolate"
Then I should see the "pancake" recipe in the search results
And I should not see the "french toast" recipe in the search results


1 scenario
6 steps passed
(commit)

I have the feeling I am missing another field or two, but that does it for the defined search scenarios. There are four more search scenarios that need Given/When/Then text descriptions (and subsequent implementation) so I will get started on that tomorrow.

No comments:

Post a Comment