Saturday, April 11, 2009

Some Recipe Detail Clean-Up

‹prev | My Chain | next›

At this point, I have the entire Recipe Details feature (5 scenarios and 20 steps) passing:
cstrom@jaynestown:~/repos/eee-code$ cucumber features/recipe_details.feature -n 
Feature: Recipe Details

So that I can accurately reproduce a recipe at home
As a web user
I want to be able to easily recognize important details
Scenario: Viewing a recipe with several ingredients
Given a recipe for Buttermilk Chocolate Chip Pancakes
When I view the recipe
Then I should see an ingredient of "1 cup all-purpose, unbleached flour"
And I should see an ingredient of "¼ teaspoons salt"
And I should see an ingredient of "chocolate chips (Nestle Tollhouse)"

Scenario: Viewing a recipe with non-active prep time
Given a recipe for Crockpot Lentil Andouille Soup
When I view the recipe
Then I should see 15 minutes of prep time
And I should see that it requires 5 hours of non-active cook time

Scenario: Viewing a list of tools used to prepare the recipe
Given a recipe for Chicken Noodle Soup
When I view the recipe
Then I should see that it requires a Bowl, a Colander, a Cutting Board, a Pot and a Skimmer to prepare

Scenario: Main site categories
Given a recipe for Mango and Tomato Salad
When I view the recipe
Then I should see the site-wide categories of Italian, Asian, Latin, Breakfast, Chicken, Fish, Meat, Salad, and Vegetarian
And the Salad and Vegetarian categories should be active

Scenario: Viewing summary and recipe instructions
Given a recipe for Curried Shrimp
When I view the recipe
Then I should a nice summary of the dish
And I should see detailed, easy-to-read instructions


5 scenarios
20 steps passed
Inside the application, I have 30 examples passing:
cstrom@jaynestown:~/repos/eee-code$ rake
(in /home/cstrom/repos/eee-code)
/usr/bin/ruby1.8 ./spec/eee_spec.rb
...

Finished in 0.073494 seconds

3 examples, 0 failures
/usr/bin/ruby1.8 ./spec/eee_helpers_spec.rb
......

Finished in 0.014949 seconds

6 examples, 0 failures
/usr/bin/ruby1.8 ./spec/views/recipe.haml_spec.rb
.......................

Finished in 0.191058 seconds

23 examples, 0 failures
(commit of the Rakefile).

With all that work done, I have yet to actually look at a recipe in a browser. Now is as good a time as any:



On first inspection, everything looks good, but I realize that I have not linked in the recipe images yet. Scrolling down, I see:



Oops, the parenthetical ingredient brands are showing even when there is no brand. I thought I had accounted for that case, but obviously not. No matter, it should not be difficult to fix. But first, the recipe photo...

Including a Recipe Photo

In the HAML spec, I am mostly interested in the location of the image in the document. The examples that describe this:
  context "a recipe with an image" do
it "should include an image in the recipe summary" do
self.stub!(:image_link).and_return("<img/>")
render("views/recipe.haml")
response.should have_selector("#eee-summary > img")
end

end

context "a recipe without an image" do
it "should not include an image" do
self.stub!(:image_link).and_return(nil)
render("views/recipe.haml")
response.should_not have_selector("#eee-summary > img")
end
end
I am stubbing out a helper method, so the actual implementation is trivial:
#eee-summary
= image_link @recipe
= wiki @recipe['summary']
When I started my red-green-refactor cycle, the responsibility for the image tag was in the HAML template. It got ugly as I started handling boundary conditions, which is how the image_link helper came into being. The URL for images was established towards the end of the recipe details feature. The examples:
describe "image_link" do
it "should return a link tag pointing to the document's image" do
doc = {
'_id' => "foo",
'_attachments' => { 'sample.jpg' => { } }
}

image_link(doc).
should have_selector("img",
:src => "/images/#{doc['_id']}/sample.jpg")
end

it "should return nil if no attachments" do
image_link({ }).should be_nil
end
it "should return nil if no image attachments" do
doc = { '_attachments' => { 'sample.txt' => { } } }
image_link(doc).should be_nil
end
end
And the code that actually implements it:
    def image_link(doc)
return nil unless doc['_attachments']

filename = doc['_attachments'].
keys.
detect{ |f| f =~ /jpg/ }

return nil unless filename

%Q|<img src="/images/#{doc['_id']}/#{filename}"/>|
end
Since I do not have a cucumber scenario, I will just check the output in the browser:

(commit)

Boundary Condition: No Brand

Investigating this, I find that I accounted for the case of when brands are nil, but not empty strings. The empty strings are an artifact of the import from the legacy Rails application. I hate to add code to account for an artifact, but this is fairly trivial. If this is the only workaround, I can live with it. A TODO can serve as a reminder in case more artifacts are found, at which point, a CouchDB migration can be used.

This would be trivial to address with Rails' blank?, but, even without it, it is not too difficult. First, the example (including the TODO):
  # TODO: this should not be necessary.  The blank brands are an
# artifact of the import-from-rails process
context "a recipe with a blank brand" do
it "should not include brand information" do
@recipe['preparations'] = []
@recipe['preparations'] << {
'quantity' => 1,
'unit' => '12 ounce bag',
'brand' => '',
'ingredient' => {
'name' => 'chocolate chips'
}
}

render("views/recipe.haml")

response.should_not have_selector(".ingredient > .brand")
end
end
To implement this example, a Regexp check for any non-space character will work:
        - if preparation['brand'] =~ /\S/
%span.brand
= "(" + preparation['brand'] + ")"
To prevent the nil brand case from breaking, this code relies on the fact that the NilClass does support Regexp matching. Specifically:
>> nil =~ /\S/
=> false
A quick check in the browser confirms that the issue has been resolved—no more blank brands:

(commit)

That should wrap up work on the recipe details. Next up, search, I think.

Friday, April 10, 2009

Oooh! Updated couchdb-lucene

‹prev | My Chain | next›

I started worrying about indexing whole documents as soon as I got couchdb-lucene working with my prototype database. By default, couchdb-lucene indexes attributes individually. This means that the index can be searched for recipes whose titles contain the word "chilled", but there is no way to search for documents that contain the word "chilled" anywhere.

That nagging concern has remained in the intervening weeks, so I was quite excited to see an example of how to do this on Robert Newson's fork of couchdb-lucene.

To get that up and running on my local copy of CouchDB 0.9, I need to update my local copy and rebuild:
cd ~/repos/couchdb-lucene
git pull
mvn
To point my local copy of CouchDB to the new version of the lucene indexer, I update etc/couchdb/local_dev.ini:
; CouchDB Configuration Settings

; Custom settings should be made in this file. They will override settings
; in default.ini, but unlike changes made to default.ini, this file won't be
; overwritten on server upgrade.

[couchdb]
;max_document_size = 4294967296 ; bytes

[httpd]
; port = 5985
;bind_address = 127.0.0.1

[log]
; level = debug

[update_notification]
;unique notifier name=/full/path/to/exe -with "cmd line arg"

[couchdb]
os_process_timeout=60000 ; increase the timeout from 5 seconds.

[external]
fti=/usr/bin/java -jar /home/cstrom/repos/couchdb-lucene/target/couchdb-lucene-0.3-SNAPSHOT-jar-with-dependencies.jar -search

[update_notification]
indexer=/usr/bin/java -jar /home/cstrom/repos/couchdb-lucene/target/couchdb-lucene-0.3-SNAPSHOT-jar-with-dependencies.jar -index

[httpd_db_handlers]
_fti = {couch_httpd_external, handle_external_req, <<"fti">>}
To choose a different indexing algorithm, a new design document is needed. In futon, choose "Design documents" from the "Select view" drop down:



Next, choose "Create Document ..." from the top of the UI, and enter "_design/lucene" for the name:



Finally add the following code (from the couchdb-lucene documentation), which indexes the entire document in the all field:
function(doc) {
var ret = new Document();

function idx(obj) {
for (var key in obj) {
switch (typeof obj[key]) {
case 'object':
idx(obj[key]);
break;
case 'function':
break;
default:
ret.field(key, obj[key]);
ret.field('all', obj[key]);
break;
}
}
}

idx(doc);

return ret;
}
To a new transform attribute (make sure to enclose it in quotes):



Finally, make sure to save the document! I always seem to forget this step which causes all sorts of confusion.

To test the full document, all search, use curl:
cstrom@jaynestown:~$ curl http://localhost:5984/eee/_fti?q=all:wheatberries
{"q":"+_db:eee +all:wheatberries",
"etag":"1202191c3d1",
"skip":0,
"limit":25,
"total_rows":1,
"search_duration":1,
"fetch_duration":0,
"rows":[{"_id":"2008-07-19-oatmeal",
"score":0.5710114240646362}]}
Just to be sure that something has not broken after the change, I check the same search on the instructions field, which pulls back the same result ("wheatberries" was mentioned in the instructions of the oatmeal recipe):
cstrom@jaynestown:~$ curl http://localhost:5984/eee/_fti?q=instructions:wheatberries
{"q":"+_db:eee +instructions:wheatberries",
"etag":"1202191c3d1",
"skip":0,
"limit":25,
"total_rows":1,
"search_duration":1,
"fetch_duration":1,
"rows":[{"_id":"2008-07-19-oatmeal",
"score":0.6242526769638062}]}

Thursday, April 9, 2009

Finishing Recipe Details

‹prev | My Chain | next›

Working my way back out to the features, I hit a snag. The main application spec exercises the new wiki helper method with nil input. The result is an error. I need to describe how the helper should work in such a situation:
  it "should return an empty string if called with nil" do
wiki(nil).should == ""
end
Making this example pass is a simple matter of using an empty string if nil is supplied to the wiki helper:
    def wiki(original)
text = (original || '').dup
text.gsub!(/\b(\d+)F/, "\\1° F")
text.gsub!(/\[kid:(\w+)\]/m) { |kid| kid_nicknames[$1] }
text.gsub!(/\[recipe:(\S+)\]/m) { |r| recipe_link($1) }
RedCloth.new(text).to_html
end
(commit)

Back Out to the Feature

Running the feature that started this helper effort reveals that I need to check off two more steps in the scenario:
cstrom@jaynestown:~/repos/eee-code$ cucumber features/recipe_details.feature \
-n -s "Viewing summary and recipe instructions"
Feature: Recipe Details

So that I can accurately reproduce a recipe at home
As a web user
I want to be able to easily recognize important details
Scenario: Viewing summary and recipe instructions
Given a recipe for Curried Shrimp
When I view the recipe
Then I should a nice summary of the dish
And I should see detailed, easy-to-read instructions


1 scenario
2 steps passed
2 steps pending (2 with no step definition)

You can use these snippets to implement pending steps which have no step definition:

Then /^I should a nice summary of the dish$/ do
end

Then /^I should see detailed, easy\-to\-read instructions$/ do
end
Those pending steps can be implemented with matchers similar to:
Then /^I should a nice summary of the dish$/ do
response.should have_selector("p", :content => "This dish is yummy") do |p|
p.should have_selector("strong", :content => "yummy")
end
end

Then /^I should see detailed, easy\-to\-read instructions$/ do
response.should have_selector("p", :content => "While the shrimp are defrosting, we chop the vegetables.")
end
(commit)

Wednesday, April 8, 2009

More Recipe Helper Specs

‹prev | My Chain | next›

Tonight I try to finish off my recipe helpers.

First up is an example for wikifying kid nicknames. This was partly implemented last night, but I wanted to pull the data from CouchDB rather than from a hard-coded hash lookup. The example I try is:
    it "should lookup kid nicknames" do
RestClient.stub!(:get).and_return('{"marsha":"the oldest Brady girl"}')
wiki("[kid:marsha]").should contain("the oldest Brady girl")
end
I am not testing the CouchDB integration point here in the helper, so I just stub it out. I try to make this example pass with this helper method:
    def wiki(original)
text = original.dup
text.gsub!(/\b(\d+)F/, "\\1° F")
text.gsub!(/\[kid:(\w+)\]/m) { |kid| kid_nicknames[$1] }
text.gsub!(/\[recipe:(\S+)\]/m) { |r| recipe_link($1) }
RedCloth.new(text).to_html
end

def kid_nicknames
@@kid_kicknames ||= JSON.parse(RestClient.get("#{@@db}/kids"))
end
Unfortunately, I get an error due to the @@db class instance variable not being defined:
1)
NameError in 'wiki data stored in CouchDB should lookup kid nicknames in CouchDB'
uninitialized class variable @@db in Eee::Helpers
./helpers.rb:32:in `kid_nicknames'
./helpers.rb:26:in `wiki'
./helpers.rb:26:in `gsub!'
./helpers.rb:26:in `wiki'
./spec/eee_helpers_spec.rb:32:
./spec/eee_helpers_spec.rb:3:
Ick. The @@db instance variable will be available in the running Sinatra application. Since I am testing helpers in isolation, I get a nice failure.

I think that this is telling me that I need to move this code into a separate class (possibly a caching class). But for now, I cheat by adding a stub-able helper method _db. The specification then requires a context to stub out that method:
  context "data stored in CouchDB" do
before(:each) do
self.stub!(:_db).and_return("http://example.org/couchdb")
end

it "should lookup kid nicknames" do
RestClient.stub!(:get).and_return('{"marsha":"the oldest Brady girl"}')
wiki("[kid:marsha]").should contain("the oldest Brady girl")
end
The code implementing this example is then:
    def kid_nicknames
@@kid_kicknames ||= JSON.parse(RestClient.get("#{_db}/kids"))
end

def _db
@@db
end
Finishing the remaining helper specs are relatively trivial after that.
(commit)

Tuesday, April 7, 2009

More Helper Specs

‹prev | My Chain | next›

A long softball game (8 walks an inning was typical), precludes me from making much progress tonight. Nevertheless I must offer up some work to the gods of #mychain, lest they take offense and smite me.

As fate would have it, I left off last night with some fairly simple helper specs pending implementation. So let's implement...

First up is that the wiki helper should convert textile to HTML. To specify this, I use an example of some very simple textile, which should be converted to HTML. The RedCloth library can take care of the full specification, I am only interested in specifying that it quacks like a textile converter:
  it "should convert textile to HTML" do
textile = <<_TEXTILE
paragraph 1 *bold text*

paragraph 2
_TEXTILE

wiki(textile).
should have_selector("p", :content => "paragraph 1 bold text") do |p|
p.should have_selector("strong", :content => "bold text")
end
end
Implementation of that is as easy as wrapping the wiki method around a RedCloth call:
    def wiki(text)
RedCloth.new(text).to_html
end
Next up is that the wiki call should wikify temperatures, which can be specified as:
  it "should wikify temperatures" do
wiki("250F").should contain("250° F")
end
This can be implemented as:
    def wiki(original)
text = original.dup
text.gsub!(/\b(\d+)F/, "\\1° F")
RedCloth.new(text).to_html
end
Seeing this capricious use of gsub immediately makes me think of the Eigenclass class article on markdown conversion.

I am doing bad things.
I am doing bad things.
I am doing bad things.

But, I must not prematurely optimize. If this does cost me in performance, I will deal with it later.

Staying focused on implementation rather than optimization, I move on to writing the example for wikifying kid's nicknames:
  it "should wikify kids names" do
self.stub!(:kid_nicknames).
and_return({"marsha" => "the oldest Brady girl"})

wiki("[kid:marsha]").should contain("the oldest Brady girl")
end
Implementing the code to make this example pass is a simple matter of another gsub:
    def wiki(original)
text = original.dup
text.gsub!(/\b(\d+)F/, "\\1° F")
text.gsub!(/\[kid:(\w+)\]/m) { |kid| kid_nicknames[$1] }
RedCloth.new(text).to_html
end

def kid_nicknames
{ }
end
There is still the matter of how to populate the kid_nicknames method. Hard-coding it will not do—there is not much point in obfuscating names if I make them publicly accessible elsewhere. I either need to make it a YAML configuration file or store it in CouchDB. I think I will ultimately choose to store it in CouchDB since that is easier to update (most days I think I have enough kids already, but you never know).

Since implementation will have to wait for tomorrow, I leave myself a note:
  it "should lookup kid nicknames in CouchDB"
and call it a day.
(commit)

Monday, April 6, 2009

Wiki Specifcations

‹prev | My Chain | next›

Realizing that I had overlooked recipe summary and instructions, I get started on them tonight. The feature scenario:
    Scenario: Viewing summary and recipe instructions

Given a recipe for Curried Shrimp
When I view the recipe
Then I should a nice summary of the dish
And I should see detailed, easy-to-read instructions
The inside view specs are straight-forward:
  it "should display the recipe's summary" do
self.stub!(:wiki).and_return("wiki #{@summary}")

render("/views/recipe.haml")
response.should have_selector("#eee-summary",
:content => "wiki #{@summary}")
end

it "should display the recipe's instructions" do
self.stub!(:wiki).and_return("wiki #{@instructions}")

render("/views/recipe.haml")
response.should have_selector("#eee-instructions",
:content => "wiki #{@instructions}")
end
The not-so-trivial part of working inside the guts of the application is that wiki helper method. In addition to textilizing text, it performs various domain-specific clean-up such as converting 250F to 250° F, replacing recipe URIs with linked titles and replacing our kids names with nicknames.

Happily, I spent quite a bit of time getting helpers into controller specs. Even better, that work makes direct helper specifications trivial:
require File.expand_path(File.dirname(__FILE__) + '/spec_helper' )

describe "wiki" do
it "should return simple text as unaltered text" do
wiki("bob").should == "bob"
end
it "should convert textile to HTML"
it "should wikify temperatures"
it "should wikify kids names"
it "should wikify recipe URIs"
end
A trivial implementation of the wiki makes this spec pass:
    def wiki(text)
text
end
(commit)

Next up: making those other 4 examples pass.

Sunday, April 5, 2009

Specifications for Images

‹prev | My Chain | next›

My first instinct in trying to specify how images should be accessed is to request them via the normal RSpec get method:
    describe 'GET /recipes/:permalink/:image' do
it "should retrieve the image" do
data = RestClient.get "#{@@db}/#{@permalink}"
recipe = JSON.parse(data)

RestClient.put "#{@@db}/#{@permalink}/sample.jpg?rev=#{recipe['_rev']}",
File.read('spec/fixtures/sample.jpeg'),
:content_type => 'image/jpeg'

get "/images/#{@permalink}/sample.jpg"
response.should be_ok
end
The RestClient.put call to attach the image is by-the-book CouchDB. Similarly, the the get call is idiomatic RSpec—but only when expecting a text-based response. When a binary response comes back, bad things happen:
cstrom@jaynestown:~/repos/eee-code$ ruby ./spec/eee_spec.rb
.F*

Pending:

eee a CouchDB recipe GET /recipes/:permalink/:image should return 404 for a non-existent image (TODO)
./spec/eee_spec.rb:55

1)
Rack::Lint::LintError in 'eee a CouchDB recipe GET /recipes/:permalink/:image should retrieve the image'
Body yielded non-string value "<lots and LOTS of binary data output>"
/home/cstrom/.gem/ruby/1.8/gems/rack-0.9.1/lib/rack/lint.rb:16:in `assert'
/home/cstrom/.gem/ruby/1.8/gems/rack-0.9.1/lib/rack/lint.rb:437:in `each'
/home/cstrom/.gem/ruby/1.8/gems/rack-0.9.1/lib/rack/lint.rb:435:in `each'
/home/cstrom/.gem/ruby/1.8/gems/rack-0.9.1/lib/rack/mock.rb:126:in `initialize'
/home/cstrom/.gem/ruby/1.8/gems/rack-0.9.1/lib/rack/mock.rb:70:in `new'
/home/cstrom/.gem/ruby/1.8/gems/rack-0.9.1/lib/rack/mock.rb:70:in `request'
/home/cstrom/.gem/ruby/1.8/gems/sinatra-0.9.1.1/lib/sinatra/test.rb:41:in `make_request'
/home/cstrom/.gem/ruby/1.8/gems/sinatra-0.9.1.1/lib/sinatra/test.rb:44:in `get'
./spec/eee_spec.rb:50:
./spec/eee_spec.rb:3:

Finished in 0.0737 seconds

3 examples, 1 failure, 1 pending
I would try to make a RestClient call directly to the Sinatra sever—if it were running. Unfortunately, it is not—RSpec merely pretends that it is running.

My ultimate solution is completely unsatisfactory to me. I spec the errors that the testing environment gives me:
    describe 'GET /recipes/:permalink/:image' do
it "should retrieve the image" do
data = RestClient.get "#{@@db}/#{@permalink}"
recipe = JSON.parse(data)

RestClient.put "#{@@db}/#{@permalink}/sample.jpg?rev=#{recipe['_rev']}",
File.read('spec/fixtures/sample.jpg'),
:content_type => 'image/jpeg'


lambda { get "/images/#{@permalink}/sample.jpg" }.
should raise_error(Rack::Lint::LintError)
end

it "should return 404 for a non-existent image" do
get "/images/#{@permalink}/sample2.jpg"
response.should be_not_found
end
end
Maybe slightly better than not spec'ing at all, but at least it allows me to implement:
get '/images/:permalink/:image' do
content_type 'image/jpeg'
begin
RestClient.get "#{@@db}/#{params[:permalink]}/#{params[:image]}"
rescue
404
end
end
(commit)