Monday, October 26, 2009

"Newbie" Feedback

I recently had the privilege to supply the challenge for the second ever Ruby Programming Challenge For Newbies. I thought it pretty cool that the challenge provoked 40+ "newbies" to submit responses. As one might expect, there was some rough Ruby, but very few butcherings of the language.

In the spirit of the code review, I tried to provide constructive feedback to all participants. Following is a summary of some common suggestions that I had to offer...

Bang Methods

Method parameters are mutable in Ruby. If you modify the value of the parameter inside a method, the original value changes. This can lead to all sorts of unexpected side-effects. There are some exceptions to this, but it is best to take no chances and treat it as a rule.

This can be a bit frightening at first, but Ruby has a convention of adding a bang (!) to the end of a method name when side-effects are expected. Many of the core methods follow this convention. For instance consider the difference between map and map!:
a = [1, 2, 3]

# Double every value in the array
>> a.map{|x| x * 2}
=> [2, 4, 6]

# The original value has not changed
>> a
=> [1, 2, 3]

# Double every value in the array with a bang method
>> a.map!{|x| x * 2}
=> [2, 4, 6]

# The original value is changed
>> a
=> [2, 4, 6]
The problem requested a solution with a method signature of average_time_of_day(times), where the times parameter was list of strings (e.g. %w{6:41am 6:51am 7:01am}). Since the method does not end with a bang, it should not modify the input (e.g. by calling a bang method like map!). A third of the submissions modified the times array inside that method—or worse, in other methods that were called by average_time_of_day.

Since average_time_of_day was the entirety of the challenge, modifying the input was not a huge problem. Still, not a good habit to start.

Enumerable

Since the challenge involved averaging times, most solutions involved summing up these times. A typical solution looked something like this:
sum = 0
i = 0
while i < times.length
time = times[i]
# parse / manipulate the time
sum = sum + time
i = i + 1
end
It is nice that Ruby provides syntax like for and while, making the transition from other languages so easy. There is a cleaner way to accomplish this in Ruby:
sum = 0
times.each do |time|
# parse / manipulate the time
parsed_time = #....
sum = sum + parsed_time
end
Eschewing while in favor of an each block eliminates the need for the tracking variable, i. With the assignment and computation of the tracking variable eliminated, there is less to distract from the actual intent of the code. Less code makes intent clearer.

The only thing still getting in the way of intent is the accumulator variable, sum. It is being initialized outside of the each block and manipulated inside the block. If you think of a block like a function, then this is akin to modifying a global variable inside a function.

Ruby has an inject method (also known as reduce and fold in Ruby 1.8.7+) to handle this:
sum = times.inject(0) do |accumulator, time|
# Manipulate the time somehow
accumulator + time
end
An inject starts with a seed value for an accumulator (sometimes called a memo). As inject iterates over each value, the last value in the block is used to seed the accumulator the next iteration. The end result of the iteration is the accumulation of each of these values. In the case of the code snippet above, it is the summation of the times.

But wait, there's more! It is possible to compact this even further...

Chaining

If you are not using bang methods, then you can chain iterators together. Let's say your solution involved mapping time strings to time objects and then sorting them, this can be accomplished via:
def average_time_of_day(times)
sorted_times = times.
map { |t| Time.parse(t) }.
sort
# Do more stuff ...
end
The sorted_times local variable now contains, well... sorted times. The time strings were mapped to actual time objects, which could then be sorted. That's just lovely!

Going back to the accumulator example, it is possible to chain together iterator to make intent crystal clear:
sum = times.
map { |str| Time.parse(str) }.
inject { |sum, t| sum + t }
It is possible to achieve even greater clarity through symbol to proc shorthand:
class String
def to_time
Time.parse(self)
end
end

def average_time_of_day(times)
sum = times.map(&:to_time).inject(&:+)
# Calculate the average ...
end
The theory behind symbol-to-proc is probably a bit beyond "newbie" scope, but still worth seeing to give an idea how close to the domain Ruby allows you to get. In that last code snippet, I am taking the times strings, mapping them to Time instances (with a monkey patch to String) and adding them all together. Intent does not get much clearer than that.

Read the documentation on Ruby's Enumerable module. Even seasoned Rubyists can benefit from re-reading it regularly.

Test Harness

Some will claim that the Ruby community is too "test obsessed". There are very good reasons to embrace TATFT / BDD, etc. I could write for months on why this is a good idea. In fact I did. But this is not one of those times.

Using a test harness for an algorithm like average_time_of_day makes too much sense not to use it. Many of the submissions were clearly iterating toward a solution. The author got it working for the simple case, then tried to get it working crossing midnight—only to break the simple case. Many authors never noticed that the simple case was no longer working and submitted half working solutions.

As soon as you have a simple solution working, write a test (if not sooner!):
def average_time_of_day(times)
# incredibly simplistic solution
end

require 'test/unit'

class TestAverageTimeOfDay < Test::Unit::TestCase
def test_simple_times
assert_equal('6:51am', average_time_of_day(%w{6:41am 6:51am 7:01am}))
end
end
If you had saved this file as average_time_of_day.rb, you could execute this test after each change to the average_time_of_day method thusly:
jaynestown% ruby average_time_of_day.rb
Loaded suite average_time_of_day
Started
.

1 test, 1 assertion, 0 failures, 0 errors
Ruthlessly running this command as the average_time_of_day method evolves will tell you immediately when something is broken. The sooner you know something is wrong, the sooner you can address it.

Don't test because a segment of the Ruby community forces you to, do it because it is so damn easy there is no reason not do.

Conclusion

That about sums up my feedback to the "newbies". I really did enjoy providing the challenge and very much appreciated so many quality solutions. I hope most of you continue your learning and I look forward to seeing some of your names driving the future of Ruby!

Friday, October 16, 2009

Smoke 'em If You Got 'em

‹prev | My Chain | next›

I did a quick push to the beta site this morning so that I could do some smoke testing. For the most part, everything looked good. Sadly, I did discover another oversight in my switch of the recipe URL delimiter from dashes to slashes. The recipe search results still had the dashes in them.

I am somewhat disappointed that my Cucumber scenarios did not catch this error. It would have been a simple matter of one of the 10 recipe search scenarios following a link from the search results to the actual recipe. A simple matter indeed, but I did not include such a step.

This is, of course, the point of smoke tests. Even if you have awesome tests, even if you do not make big changes the day before deploying, it is always a good idea to poke around. There may be a bit of smoke, there may be a raging inferno, but you won't know unless you exercise the code.

After fixing the link in the search results (a minor code change, but involving many spec changes), I am ready to deploy. But first...

I am not certain that I have identified the last of the dash/slash issues. Rather than delay the promotion to production, I do a bit of defensive Rack configuration with Rack::Rewrite. Specifically, a recipe URL that contains dashes should be redirected to the proper slash URL:
use Rack::Rewrite do
r301 %r{(.+)\.html}, '$1'
r301 %r{^/recipes/(\d{4})-(\d{2})-(\d{2})-(.+)}, '/recipes/$1/$2/$3/$4'
end
The curl command verifies that this works:
jaynestown% curl -I http://localhost:3000/recipes/2008-07-13-sausage   
HTTP/1.1 301 Moved Permanently
Location: /recipes/2008/07/13/sausage
Content-Length: 14
Connection: keep-alive
Server: thin 1.2.2 codename I Find Your Lack of Sauce Disturbing
With all of my RSpec tests passing, all of my Cucumber scenarios passing, and a bit of defensive Rack middleware, I am ready to deploy my Sinatra / CouchDB application:
jaynestown% rake vlad:stop_app
jaynestown% rake vlad:update vlad:migrate vlad:start_app
Finally, I update DNS so that http://legacy.eeecooks.com points to the legacy Rails site and http://www.eeecooks.com points to the new Sinatra / CouchDB site.

Thus endeth the chain.

Thursday, October 15, 2009

A Slight Rewrite

‹prev | My Chain | next›

As mentioned yesterday, my last minute decision to switch the dates in recipe URLs from dash separated (/recipes/YYYY-MM-DD-short_name) to slash separated (/recipes/YYYY/MM/DD/short_name) caused many things to break. My specs / unit tests caught a few of these failures. My Cucumber scenarios caught many. I am perfectly comfortable with this.

My specs drive design. Through behavior driven development, they ensure that my design is the simplest thing that can possibly work. Simple means fewer bugs today and less legacy tomorrow. And, hey, if they happen to catch a bug or two, that's great. Specs are not for regression testing, they remain to reflect my thought process when I return months (years) later.

This is not to say that regression testing is not important—it is. However, regression testing is not as valuable as clean design is for long term health of a project.

One of the aspects of Cucumber that I value so much is that it can drive features, but also verify them once they are implemented. As such, I rely heavily on Cucumber to catch regressions.

In the case of my change yesterday, I ended up with 10 regressions identified by Cucumber. I am able to fix all of them by updating visit statements in the Cucumber steps and updating a few helpers:
jaynestown% cucumber features
...
39 scenarios (1 pending, 38 passed)
344 steps (1 pending, 343 passed)
0m42.922s
One last thing for me to do is to accommodate really old URLs. When we first put up the site, we baked HTML from XML. Thus all of the URLs had .html extensions on them. The legacy Rails site supported extensions, but linked internally without the extension. But what if someone has a really old bookmark? I would prefer not to dump them onto the Not Found page and, thanks to John Trupiano's Rack::Rewrite, I do not have to worry.

After gem installing rack-rewrite, I add this to my rackup file:
require 'rack-rewrite'

###
# Rewrite
use Rack::Rewrite do
r301 %r{(.+)\.html}, '$1'
end
Now, when I ask for a sausage recipe with the .html extension, I am redirected to its new, permanent location:
jaynestown% curl -I http://localhost:3000/recipes/2008/07/13/sausage.html
HTTP/1.1 301 Moved Permanently
Location: /recipes/2008/07/13/sausage
Content-Length: 14
Connection: keep-alive
Server: thin 1.2.2 codename I Find Your Lack of Sauce Disturbing
Tomorrow I will deploy and close out my chain.

Wednesday, October 14, 2009

Regression Testing

‹prev | My Chain | next›

Gah! A discussion at B'more on Rails got me to thinking. If I am replacing my old site, will old URLs still work? It is an important question to answer as Google and others will have links to the legacy URLs. I would like to make the transition as easy as possible on my users.

I was pretty consistent with my URLs. This is the third time that I have implemented the same site, so the URLs are somewhat ingrained by now. But...
get '/recipes/:permalink' do
data = RestClient.get "#{@@db}/#{params[:permalink]}"
#...
end
The :permalink in the URL is being passed thru to CouchDB. The IDs in CouchDB are of the form YYYY-MM-DD-short_name. Thus, to pull back a recipe, I would need to access a URL in the Sinatra app of the form /recipes/YYYY-MM-DD-short_name.

That all works well. It is fully tested and even verified with Cucumber. The problem? The legacy site links to URLs like /recipes/YYYY/MM/DD/short_name. Slightly less bothersome, the rest of the Sinatra app is consistent using slashes rather than dashes. So, to keep consistent with the rest of the site and the legacy site, I need to switch the Sinatra app to:
get %r{/recipes/(\d+)/(\d+)/(\d+)/?(.*)} do |year, month, day, short_name|
data = RestClient.get "#{@@db}/#{year}-#{month}-#{day}-#{short_name}"
#...
end
I do not know if it is hubris, but I like to make these kinds of changes without testing first. I already have excellent test coverage, I expect those tests to identify problems in the code caused by such a change. Put another way, this is already covered by tests, I just need to update the expectations to align with a new reality.

Running all of my specs, I find two such failures:
jaynestown% spec ./spec/eee_spec.rb
................FF.....................................

1)
Spec::Mocks::MockExpectationError in 'eee cached documents GET /recipes/permalink should etag with the CouchDB document's revision'
RestClient expected :get with (any args) once, but received it 0 times
./spec/eee_spec.rb:243:

2)
'eee a CouchDB recipe GET /recipes/permalink should respond OK' FAILED
expected ok? to return true, got false
./spec/eee_spec.rb:275:

Finished in 1.50 seconds
Both are easily fixed by changing the expectation to honor the new slash URLs.

So is such a change just that easy? Not quite.

Tests / examples are useful for driving design. They help to ensure that I implement the simplest thing that could possibly work. This, in turn, allows for future growth. Because I implement the simplest thing that can possibly work, I am all but certain to be adhering to YAGNI (You Ain't Gonna Need It). In other words, I am not painting myself into a corner designing some super elegant architecture only find 3 months later that I need to remove it because it is causing problems or worse, preventing from moving in a different, previously unanticipated direction.

So tests / examples are invaluable to me, but they do not do a good job of finding bugs. So what about regression testing? Well, that is what Cucumber's full stack execution is great for. And right now, I have 12 failing scenarios where once I had all passing:

jaynestown% cucumber features
...
Failing Scenarios:
cucumber features/recipe_replacement.feature:14 # Scenario: A previous version of the recipe
cucumber features/recipe_details.feature:7 # Scenario: Viewing a recipe with several ingredients
cucumber features/recipe_details.feature:15 # Scenario: Viewing a recipe with non-active prep time
cucumber features/recipe_details.feature:22 # Scenario: Viewing a list of tools used to prepare the recipe
cucumber features/recipe_details.feature:28 # Scenario: Main site categories
cucumber features/recipe_details.feature:35 # Scenario: Viewing summary and recipe instructions
cucumber features/recipe_details.feature:42 # Scenario: Navigating to other recipes
cucumber features/recipe_alternate_preparations.feature:14 # Scenario: Alternate preparation
cucumber features/draft_recipes.feature:7 # Scenario: Navigating between recipes
cucumber features/browse_meals.feature:36 # Scenario: Browsing a meal on a specific date
cucumber features/site.feature:7 # Scenario: Quickly scanning meals and recipes accessible from the home page
cucumber features/site.feature:64 # Scenario: Send compliments to the chef on a delicious recipe

39 scenarios (12 failed, 1 pending, 26 passed)
344 steps (12 failed, 37 skipped, 1 pending, 294 passed)
0m40.568s
The problem is twofold: some Cucumber steps are visiting recipe URLs with slashes and I have helpers that are still generating links to recipes using the slashes. The former is easy enough to address. An example of the latter is this failure:
jaynestown% cucumber features/browse_meals.feature:36
Sinatra::Test is deprecated; use Rack::Test instead.
Feature: Browse Meals

So that I can find meals made on special occasions
As a person interested in exploring meals and how they drive certain recipes
I want to browse meals by date

Scenario: Browsing a meal on a specific date # features/browse_meals.feature:36
Given a "Focaccia" recipe from March 3, 2009 # features/step_definitions/recipe_details.rb:137
Given a "Focaccia! The Dinner" meal with the "Focaccia" recipe on the menu # features/step_definitions/meal_details.rb:23
When I view the "Focaccia! The Dinner" meal # features/step_definitions/meal_details.rb:54
Then I should see the "Focaccia! The Dinner" title # features/step_definitions/meal_details.rb:71
And I should see a "Focaccia" recipe link in the menu # features/step_definitions/meal_details.rb:91
When I click the "March" link # features/step_definitions/meal_details.rb:63
Then I should see "Focaccia! The Dinner" in the list of meals # features/step_definitions/meal_details.rb:103
When I click the "Focaccia! The Dinner" link # features/step_definitions/meal_details.rb:63
And I click the "2009" link # features/step_definitions/meal_details.rb:63
Then I should see "Focaccia! The Dinner" in the list of meals # features/step_definitions/meal_details.rb:103
When I click the "Focaccia! The Dinner" link # features/step_definitions/meal_details.rb:63
When I click the "Focaccia" link # features/step_definitions/meal_details.rb:63
Then I should see the "Focaccia" recipe # features/step_definitions/meal_details.rb:107
expected following output to contain a <h1>Focaccia</h1> tag:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
<title>EEE Cooks</title>
<link href="/stylesheets/style.css" rel="stylesheet" type="text/css">
<link href="main.rss" rel="alternate" title="EEE Cooks RSS" type="application/rss+xml">
<link href="recipes.rss" rel="alternate" title="EEE Cooks Recipe RSS" type="application/rss+xml">
</head>
<html><body>
<div id="header">
<div id="eee-header-logo">
<a href="/">
<img alt="Home" src="/images/eee_corner.png"></a>
</div>
</div>
<ul id="eee-categories">
<li><a href="/recipes/search?q=category:italian">Italian</a></li>
...
<h1>
Not Found
</h1>

...
The link from the meal includes the slashes. When Cucumber tries to follow that link, the result is a page not found.

A gsub in the recipe_link helper addresses that problem:
    def recipe_link(link, title=nil)
permalink = link.gsub(/\//, '-')
recipe = JSON.parse(RestClient.get("#{_db}/#{permalink}"))
%Q|<a href="/recipes/#{recipe['_id'].gsub(/-/, '/')}">#{title || recipe['title']}</a>|
end
After updating the spec to align with the new reality of the code, I have that scenario passing as well as one other:
jaynestown% cucumber features                         
...
Failing Scenarios:
cucumber features/recipe_replacement.feature:14 # Scenario: A previous version of the recipe
cucumber features/recipe_details.feature:7 # Scenario: Viewing a recipe with several ingredients
cucumber features/recipe_details.feature:15 # Scenario: Viewing a recipe with non-active prep time
cucumber features/recipe_details.feature:22 # Scenario: Viewing a list of tools used to prepare the recipe
cucumber features/recipe_details.feature:28 # Scenario: Main site categories
cucumber features/recipe_details.feature:35 # Scenario: Viewing summary and recipe instructions
cucumber features/recipe_details.feature:42 # Scenario: Navigating to other recipes
cucumber features/recipe_alternate_preparations.feature:14 # Scenario: Alternate preparation
cucumber features/draft_recipes.feature:7 # Scenario: Navigating between recipes
cucumber features/site.feature:64 # Scenario: Send compliments to the chef on a delicious recipe

39 scenarios (10 failed, 1 pending, 28 passed)
344 steps (10 failed, 27 skipped, 1 pending, 306 passed)
0m41.200s
(commit)

Most, if not all, of the remaining failures are caused by bad Cucumber visit steps. I will address those tomorrow. Then I will be done with my chain.

Tuesday, October 13, 2009

404 in Sinatra

‹prev | My Chain | next›

Before going live, we need legit 404 and 500 error messages. In production mode, the default Sinatra error messages are a little unhelpful:



Sinatra provides two code blocks for handling errors:
not_found do
end

error do
end
I like having my not found / error templates to be named 404 / 500. Since I am using Haml, I can use the 404.haml and 500.haml templates:
not_found do
haml :'404'
end

error do
haml :'500'
end
I use :'404' because the haml method requires a symbol and :404 is not a valid symbol. After defining the contents of the Haml templates, I have my 404 and 500 error pages.

The only other thing that I do tonight is define the kid nicknames document. We share a lot of personal information about our family in the cookbook, but never the actual kids' names (we're even a little vague about birthdays). Maybe we're paranoid, but...

I have a helper that pulls the nicknames from CouchDB:
    def kid_nicknames
@@kid_kicknames ||= JSON.parse(RestClient.get("#{_db}/kids"))
end
Those nicknames are then used as part of the "wiki" method:
    def wiki(original, convert_textile=true)
text = (original || '').dup
#...
text.gsub!(/\[kid:(\w+)\]/m) { |kid| kid_nicknames[$1] }
#...
end
The seed data for the kids kids document has been empty. I define it:
{
"___":"our daughter",
"___":"our son",
"___":"the baby"
}
And then use my couch_docs gem to load into the CouchDB database:
jaynestown% couch-docs load . http://localhost:5984/eee
With that, I have the nicknames displaying correctly:



That will do for tonight. Tomorrow, I deploy the 404/500 and seed data to beta. Then I can switch beta to be the production site, thus ending my chain.

Monday, October 12, 2009

Deploying ImageScience to Debian

‹prev | My Chain | next›

After cleaning up a few minor issues, I have all of my RSpec examples passing:
jaynestown% rake                                      
(in /home/cstrom/repos/eee-code)

==
Sinatra app spec
.......................................................

Finished in 1.89 seconds

55 examples, 0 failures

==
Helper specs
..........................................................................................

Finished in 0.13 seconds

90 examples, 0 failures

==
View specs
..............................................................................................................

Finished in 1.37 seconds

110 examples, 0 failures
All of the Cucumber scenarios are passing as well:

jaynestown% cucumber features
...
39 scenarios (1 pending, 38 passed)
344 steps (1 pending, 343 passed)
0m43.335s
I have my various Rack middlewares configured so I think I am ready to deploy to the beta site:
jaynestown% rake vlad:stop_app
jaynestown% rake vlad:update vlad:migrate vlad:start_app
I run into a small problem with ImageScience and RubyInline. Specifically, I see errors similar to:
/home/cstrom/.ruby_inline/Inline_ImageScience_cdab.c:2:23: error: FreeImage.h: No such file or directory
/home/cstrom/.ruby_inline/Inline_ImageScience_cdab.c: In function ‘unload’:
/home/cstrom/.ruby_inline/Inline_ImageScience_cdab.c:8: error: ‘FIBITMAP’ undeclared (first use in this function)
/home/cstrom/.ruby_inline/Inline_ImageScience_cdab.c:8: error: (Each undeclared identifier is reported only once
...
I ultimately track that down to a missed system level package. I use Ubuntu for my development box, but Debian for the deployment server. On the former, the freeimage development packages was libfreeimage3-dev, but on Debian it should be libfreeimage-dev (without the 3). I had missed the no-such package warning in the apt-get output. No doubt this is the kind of thing that Chef would address.

At any rate, I now have the thumbnailing site up and running on the beta site, and the images are nice and small on the homepage:



I still need a proper 404 page and one last bit of seed data. Then I am done with my chain.

Sunday, October 11, 2009

Rack::ThumbNailer

‹prev | My Chain | next›

As of yesterday, I have my Rack middleware thumbnail-ing application images when the request has the thumbnail parameter.

I am using ImageScience (documentation), which does all of its work in the filesystem. Since thumbnails are being stored on the filesystem, I might as well keep them there and serve them up whenever subsequent requests come in for the same resource.

Describing this in RSpec:
      context "and a previously generated thumbnail" do
before(:each) do
File.stub!(:exists?).and_return(true)
end
it "should not make a new thumbnail" do
Rack::ThumbNailer.
should_not_receive(:mk_thumbnail)
get "/foo.jpg", :thumbnail => 1
end
end
This fails because nothing is preventing the call to mk_thumbnailer:
1)
Spec::Mocks::MockExpectationError in 'ThumbNailer Accessing an image with thumbnail param and a previously generated thumbnail should not make a new thumbnail'
expected :mk_thumbnail with ("/var/cache/rack/thumbnails/foo.jpg", "image data") 0 times, but received it once
./thumbnailer.rb:18:in `call'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/mock_session.rb:30:in `request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:207:in `process_request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:57:in `get'
./thumbnailer_spec.rb:67:
To make that example pass, I add an unless File.exists? to the call Rack method:
    def call(env)
req = Rack::Request.new(env)
if !req.params['thumbnail'].blank?
filename = @options[:cache_dir] + req.path_info

unless ::File.exists?(filename)
image = ThumbNailer.rack_image(@app, env)
ThumbNailer.mk_thumbnail(filename, image)
end

thumbnail = ::File.new(filename).read
[200, { }, thumbnail]
else
@app.call(env)
end
end
That should just about do it. I add it to my config.ru Rackup file:
...
###
# Thumbnail

require 'rack/thumbnailer'
use Rack::ThumbNailer,
:cache_dir => '/tmp/rack/thumbnails'

###
# Sinatra App
...
Trying this out for real, I access a real image:



And, when I add the thumbnail query parameter:



Nice!

Now that I have my Rack middleware, I need to use it. I have designed it such that the image needs to be called with a query parameter. Normally, a query parameter on an image request will have no effect. Thus, opting for a query parameter seemed like a nice way of keeping my Sinatra app somewhat de-coupled from the Rack middleware. If the middleware is present a thumbnail will be returned. If the Rack middleware is not present, the full-size image will be returned (possibly with width and height attributes scaling the image in the browser).

The homepage currently has meal thumbnails included like:
...
%a{:href => date.strftime("/meals/%Y/%m/%d")}
= (image_link meal, :width => 200, :height => 150)
%h2
%a{:href => date.strftime("/meals/%Y/%m/%d")}= meal["title"]
...
Currently the image_link helper is returning an image tag that tells the browser to scale the image (something like <img src="/images/foo.jpg" width="200" height="150">). I would like to be able to pass additional query parameters to the image_link helper. I describe this in RSpec as:
describe "image_link" do
it "should include query parameters" do
image_link(@doc, { }, :foo => 1).
should have_selector("img",
:src => "/images/#{@doc['_id']}/sample.jpg?foo=1")
end
end
At first that fails with an incorrect number of arguments message:
1)
ArgumentError in 'image_link a document with an image attachment should include query parameters'
wrong number of arguments (3 for 2)
./spec/eee_helpers_spec.rb:202:in `image_link'
./spec/eee_helpers_spec.rb:202:
I change the message by adding an additional, optional third argument to the helper:
    def image_link(doc, options={ }, query_params={ })
# ...
end
I still get a failure because the expectation is not met:
1)
'image_link a document with an image attachment should include query parameters' FAILED
expected following output to contain a <img src='/images/foo/sample.jpg?foo=1'/> tag:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html><body><img src="/images/foo/sample.jpg"></body></html>
./spec/eee_helpers_spec.rb:202:
To make it pass, I rely on Rack::Utils.build_query to assemble the query string from a hash:
    def image_link(doc, options={ }, query_params={ })
#...
attrs = options.map{|kv| %Q|#{kv.first}="#{kv.last}"|}.join(" ")
query = query_params.empty? ? "" : "?" + Rack::Utils.build_query(query_params)
%Q|<img #{attrs} src="/images/#{doc['_id']}/#{filename}"/>|
end
With that passing, I update the homepage to trigger a thumbnail, if the Rack middleware is present:
          %a{:href => date.strftime("/meals/%Y/%m/%d")}
= (image_link meal, {:width => 200, :height => 150}, {:thumbnail => 1})
%h2
%a{:href => date.strftime("/meals/%Y/%m/%d")}= meal["title"]
And, checking it out in Firebug, the load time for my images is indeed down from 1-2 seconds all the way to ~100ms:



I will get that deployed to the beta site tomorrow and then... I think my chain may be done!

Saturday, October 10, 2009

Rack Middleware: Driven Almost to Completion

‹prev | My Chain | next›

Continuing the unexpectedly long implementation of my Rack::ThumbNailer Rack middleware, I next describe that the middleware needs to pull images from the target application:
      it "should pull the image from the target application" do
Rack::ThumbNailer.should_receive(:rack_image)
get "/foo.jpg", :thumbnail => 1
end
I make the rack_image another class method due to my inability to stub the target application. To make that pass:
    def call(env)
req = Rack::Request.new(env)
if !req.params['thumbnail'].blank?
filename = "foo"

ThumbNailer.rack_image(@app, env)
ThumbNailer.mk_thumbnail(filename)

thumbnail = ::File.new(filename).read
[200, { }, thumbnail]
else
@app.call(env)
end
end
When creating the thumbnail file, the mk_thumbnail class method should use the result of ThumbNailer.rack_image. To describe this in RSpec, I add a default output for ThumbNailer.rack_image in the before(:each) setup block:
    context "with thumbnail param" do
before(:each) do
#...
Rack::ThumbNailer.
stub!(:rack_image).
and_return("image data")
end
I can then specify that mk_thumbnail should use the return value of rack_image ("image data"):
      it "should generate a thumbnail" do
Rack::ThumbNailer.
should_receive(:mk_thumbnail).
with(anything(), "image data")
get "/foo.jpg", :thumbnail => 1
end
To make that pass, I add a second argument in the call method:
    def call(env)
req = Rack::Request.new(env)
if !req.params['thumbnail'].blank?
filename = "foo"

image = ThumbNailer.rack_image(@app, env)
ThumbNailer.mk_thumbnail(filename, image)

thumbnail = ::File.new(filename).read
[200, { }, thumbnail]
else
@app.call(env)
end
end
Rather than stick with the anthing() matcher in the mk_thumbnail example, I can specify the filename for the cached copy of the image. It should be a concatenation of the cache directory, plus the path_info for the image in the target application.

To support this, I add a default cache directory to the constructor for Rack::ThumbNailer:
  class ThumbNailer
DEFAULT_CACHE_DIR = '/var/cache/rack/thumbnails'

def initialize(app, options = { })
@app = app
@options = {:cache_dir => DEFAULT_CACHE_DIR}.merge(options)
end
...
The make-the-thumbnail-file example can then read:
      it "should generate a thumbnail" do
Rack::ThumbNailer.
should_receive(:mk_thumbnail).
with("/var/cache/rack/thumbnails/foo.jpg", "image data")
get "/foo.jpg", :thumbnail => 1
end
To make that pass, I update the call method to:
    def call(env)
req = Rack::Request.new(env)
if !req.params['thumbnail'].blank?
filename = @options[:cache_dir] + req.path_info

image = ThumbNailer.rack_image(@app, env)
ThumbNailer.mk_thumbnail(filename, image)

thumbnail = ::File.new(filename).read
[200, { }, thumbnail]
else
@app.call(env)
end
end
Nice, that is a full-featured thumbnail generating bit of Rack middleware, save for the two class methods, which I largely borrow from my spike the other day (I deleted the code, but kept the notes):
    private
def self.rack_image(app, env)
http_code, headers, body = app.call(env)

img_data = ''
body.each do |data|
img_data << data
end
img_data
end


def self.mk_thumbnail(filename, image_data)
path = filename.sub(/\/[^\/]+$/, '')
FileUtils.mkdir_p(path)

ImageScience.with_image_from_memory(image_data) do |img|
img.resize(200, 150) do |small|
small.save cache_filename
end
end
end
I would prefer to test these as well, but my inability to set expectations / mock these instance method makes such an endeavor more trouble that it is worth. So I keep them small, focused and rely on the axiom that one does not test private methods.

Tomorrow, I will add cache hit/miss functionality and that should finish of my middleware.

Friday, October 9, 2009

Can't Stub Rack

‹prev | My Chain | next›

In my Rack application, I will have a private method that generates thumbnails. An RSpec example that describes this:
      it "should generate a thumbnail" do
app.should_receive(:mk_thumbnail)
get "/foo.jpg", :thumbnail => 1
end
In that example, app is the application defined for the benefit of Rack::Test:
def app
target_app = mock("Target Rack Application")
target_app.
stub!(:call).
and_return([200, { }, "Target app"])

Rack::ThumbNailer.new(target_app)
end
The example fails with:
1)
Spec::Mocks::MockExpectationError in 'ThumbNailer Accessing an image with thumbnail param should generate a thumbnail'
# expected :mk_thumbnail with (any args) once, but received it 0 times
./thumbnailer_spec.rb:44:

Finished in 0.015466 seconds

4 examples, 1 failure
To make that pass, I add the private method mk_thumbnail and a call to mk_thumbnail:
    def call(env)
req = Rack::Request.new(env)
if !req.params['thumbnail'].blank?
filename = "foo"
mk_thumbnail(filename, env)
thumbnail = ::File.new(filename).read
[200, { }, thumbnail]
else
@app.call(env)
end
end

private
def mk_thumbnail(filename, env)
end
That should make the example pass, but:
1)
Spec::Mocks::MockExpectationError in 'ThumbNailer Accessing an image with thumbnail param should generate a thumbnail'
# expected :mk_thumbnail with (any args) once, but received it 0 times
./thumbnailer_spec.rb:44:
Hunh? That totally should have been called!

After *much* experimentation, I finally conclude / guess that the app function must be getting evaluated in a different scope than the spec. Thus, when I place the expectation on the app in the code, I am placing it on a different app than Rack::Test is actually using. I must investigate this further, because I can see needing to do this often.

For tonight though, I side-step the issue, if only to make a little bit of progress. Rather than putting the expectation on an instance of the Rack::Thumbnailer class, I convert the method to a class method and put the expectation on the class itself:
      it "should generate a thumbnail" do
Rack::ThumbNailer.should_receive(:mk_thumbnail)
get "/foo.jpg", :thumbnail => 1
end
Updating the code so that this is a class method makes this example pass:
    def call(env)
req = Rack::Request.new(env)
if !req.params['thumbnail'].blank?
filename = "foo"
Thumbnailer.mk_thumbnail(filename, env)
thumbnail = ::File.new(filename).read
[200, { }, thumbnail]
else
@app.call(env)
end
end

private
def self.mk_thumbnail(filename, env)
end
I'll call it a night there and hope to make more progress tomorrow.

Thursday, October 8, 2009

Two More Examples to Drive Rack Middleware

‹prev | My Chain | next›

From yesterday, I have the skeleton of Rack middleware:
module Rack
class ThumbNailer
def initialize(app)
@app = app
end
def call(env)
@app.call(env)
end
end
end
Right now, that doesn't do much—it simply passes thru the results of the target Rack application (my Sinatra application in this case). This is exactly what I described in RSpec examples—for most resources the middleware should just pass-thru the application results.

Even for most images in the application, the image should be passed thru with no modifications:
  context "Accessing an image" do
context "without thumbnail param" do
it "should return image directly from the target app" do
get "/foo.jpg"
last_response.body.should contain("Target app")
end
end
end
This new example still passes since the ThumbNailer middleware only passes-thru.

But, if a :thumbnail parameter is supplied, I want a thumbnail image. From my prototype, I know that ImageScience needs to save this to the file system. To return that image, I will have to create a new File instance and read from it:
  context "Accessing an image" do
context "with thumbnail param" do
before(:each) do
file = mock("File", :read => "Thumbnail")

File.
stub!(:new).
and_return(file)
end
end
In real life, the read will lift image data, for the mock, I read the string "Thumbnail" for testing. Specifically:
      it "should return a thumbnail" do
get "/foo.jpg", :thumbnail => 1
last_response.body.should contain("Thumbnail")
end
To make that pass, I need access to the parameters. The easiest way to accomplish that is via Rack::Request object. With that I can create a conditional based on the presence of the :thumbnail parameter:
    def call(env)
req = Rack::Request.new(env)
if !req.params['thumbnail'].blank?
filename = "foo"
thumbnail = File.new(filename).read
[200, { }, thumbnail]
else
@app.call(env)
end
end
I am using a dummy value for the filename here—a future example will drive the actual storage location. This example is only driving that a file is read and that is the simplest thing that can possibly work. Except:
1)
NoMethodError in 'ThumbNailer Accessing an image with thumbnail param should return a thumbnail'
undefined method `read' for #
./thumbnailer.rb:12:in `call'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/mock_session.rb:30:in `request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:207:in `process_request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:57:in `get'
./thumbnailer_spec.rb:46:
Ack! I forgot that I am working inside a module. I really want to access the top-level File class, not something encapsulated in the Rack:: namespace. Leading double-colons to the rescue:
    def call(env)
req = Rack::Request.new(env)
if !req.params['thumbnail'].blank?
filename = "foo"
thumbnail = ::File.new(filename).read
[200, { }, thumbnail]
else
@app.call(env)
end
end
With that example passing, I call it a night. Tomorrow I will pick back up with BDDing this bit of Rack middleware into submission.

Wednesday, October 7, 2009

Describing Rack with BDD

‹prev | My Chain | next›

For the past couple of nights, I have been spiking a bit of Rack middleware that thumbnails images. A conceptual overview of the flow:
              | Inbound                  ^  Response              
| Request | (possibly altered
| | by the ThumbNailer)
| |
+----v--------------------------+-----+
| |
| ThumbNailer |
| Rack Middleware |
| |
| |
| |
| |
+----+---------------------------^----+
| |
| |
+----v---------------------------+----+
| |
| Target App |
| (more Rack Middleware or |
| a Rack application) |
| |
| |
| |
+-------------------------------------+
Having deleted the code, tonight I do it right, driving the design by example.

When testing middleware, Rack::Test appreciates an app method that describes the middleware that wraps another Rack application. The "other" application is the kind of thing that mocks were made for. The beautiful thing about Rack is that any Rack component only need respond to a :call method (taking an env argument), which returns an array of HTTP status code, HTTP headers, and the body (something that responds to :each).

Thus, a target Rack application can be mocked, in RSpec, as:
def app
@target_app = mock("Target Rack Application")
@target_app.
stub!(:call).
and_return([200, { }, "Target app"])

Rack::ThumbNailer.new(@target_app)
end
With that, I can start describing how the middleware should behave. First up, the case in which the request is accessing a non-image resource:
context "Accessing a non-image resource" do
it "should return the target app" do
get "/foo"
last_response.body.should contain("Target app")
end
end
I have yet to create the ThumbNailer class, so running this example yields this failure:
jaynestown% spec ./thumbnailer_spec.rb 
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- thumbnailer (LoadError)
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from ./thumbnailer_spec.rb:3
from /home/cstrom/.gem/ruby/1.8/gems/rspec-1.1.12/lib/spec/runner/example_group_runner.rb:15:in `load'
from /home/cstrom/.gem/ruby/1.8/gems/rspec-1.1.12/lib/spec/runner/example_group_runner.rb:15:in `load_files'
from /home/cstrom/.gem/ruby/1.8/gems/rspec-1.1.12/lib/spec/runner/example_group_runner.rb:14:in `each'
from /home/cstrom/.gem/ruby/1.8/gems/rspec-1.1.12/lib/spec/runner/example_group_runner.rb:14:in `load_files'
from /home/cstrom/.gem/ruby/1.8/gems/rspec-1.1.12/lib/spec/runner/options.rb:94:in `run_examples'
from /home/cstrom/.gem/ruby/1.8/gems/rspec-1.1.12/lib/spec/runner/command_line.rb:9:in `run'
from /home/cstrom/.gem/ruby/1.8/gems/rspec-1.1.12/bin/spec:4
from /home/cstrom/.gem/ruby/1.8/bin/spec:19:in `load'
from /home/cstrom/.gem/ruby/1.8/bin/spec:19
Starting the change-the-message or make-it-pass cycle, I change the message by defining the class:
module Rack
class ThumbNailer
end
end
The example now produces this output when run:
jaynestown% spec ./thumbnailer_spec.rb
F

1)
ArgumentError in 'Accessing a non-image resource should return the target app'
wrong number of arguments (1 for 0)
./thumbnailer_spec.rb:11:in `initialize'
./thumbnailer_spec.rb:11:in `new'
./thumbnailer_spec.rb:11:in `app'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test/methods.rb:31:in `build_rack_mock_session'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test/methods.rb:27:in `rack_mock_session'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test/methods.rb:42:in `build_rack_test_session'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test/methods.rb:38:in `rack_test_session'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test/methods.rb:46:in `current_session'
./thumbnailer_spec.rb:18:

Finished in 0.007368 seconds

1 example, 1 failure
This failure is telling me that my middleware needs to have an initializer that accepts one argument—the target Rack application (the mock in app or my Sinatra app in real life):
module Rack
class ThumbNailer
def initialize(app)
@app = app
end
end
end
The message has now changed to:
jaynestown% spec ./thumbnailer_spec.rb
F

1)
NoMethodError in 'Accessing a non-image resource should return the target app'
undefined method `call' for #
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/mock_session.rb:30:in `request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:207:in `process_request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:57:in `get'
./thumbnailer_spec.rb:18:

Finished in 0.082298 seconds

1 example, 1 failure
Now my example is telling me that my Rack middleware needs to support a call method:
    def call
end
The message now tells me that any Rack application needs to have a call method that accepts an argument (the current environment):
jaynestown% spec ./thumbnailer_spec.rb
F

1)
ArgumentError in 'Accessing a non-image resource should return the target app'
wrong number of arguments (1 for 0)
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/mock_session.rb:30:in `call'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/mock_session.rb:30:in `request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:207:in `process_request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:57:in `get'
./thumbnailer_spec.rb:18:

Finished in 0.010089 seconds

1 example, 1 failure
I update the call method to accept a single argument:
    def call(env)
end
This changes message—this time telling me that the call method in a Rack application has to return something that responds to each:
jaynestown% spec ./thumbnailer_spec.rb
F

1)
NoMethodError in 'Accessing a non-image resource should return the target app'
undefined method `each' for nil:NilClass
/home/cstrom/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/mock.rb:120:in `initialize'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/mock_session.rb:31:in `new'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/mock_session.rb:31:in `request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:207:in `process_request'
/home/cstrom/.gem/ruby/1.8/gems/rack-test-0.5.0/lib/rack/test.rb:57:in `get'
./thumbnailer_spec.rb:18:

Finished in 0.009865 seconds

1 example, 1 failure
To change the message in this case, I need to reply with an array, including the HTTP status, a hash of the HTTP headers, and the content of the response:
    def call(env)
[200, { }, "Foo"]
end
Here, I finally reach a point that my middleware supports the full Rack specification. This failure is because I am returning a hard-coded string ("Foo") rather than the output of the target application:
jaynestown% spec ./thumbnailer_spec.rb
F

1)
'Accessing a non-image resource should return the target app' FAILED
expected the following element's content to include "Target app":
Foo
./thumbnailer_spec.rb:21:

Finished in 0.011375 seconds

1 example, 1 failure
To change this message, I only need change the return value. Recalling that the @app being initialized is the mock Rack object, and that, in this case, I want the middleware to pass-thru the request directly to the target (mock) Rack application, I can make the example pass with:
module Rack
class ThumbNailer
def initialize(app)
@app = app
end
def call(env)
@app.call(env)
end
end
end
At this point, the complete example reads:
require 'spec'
require 'rack/test'
require 'thumbnailer'
require 'webrat'

def app
@target_app = mock("Target Rack Application")
@target_app.
stub!(:call).
and_return([200, { }, "Target app"])

Rack::ThumbNailer.new(@target_app)
end

context "Accessing a non-image resource" do
include Rack::Test::Methods
include Webrat::Matchers

it "should return the target app" do
get "/foo"
last_response.body.should contain("Target app")
end
end
When I execute it, I get:
jaynestown% spec ./thumbnailer_spec.rb -cfs

Accessing a non-image resource
- should return the target app

Finished in 0.027749 seconds

1 example, 0 failures
At this point, I have BDD'd my way through the entire Rack specification and have a valid Rack middleware component. Tomorrow, I will continue to drive the middleware to something more useful than a simple pass-thru.

Tuesday, October 6, 2009

Rack Image Science: A Spike

‹prev | My Chain | next›

I started a spike last night to help me understand what was needed to thumbnail images in Rack. The idea is that, given a request for an image ending in _sm.jpg, the handler should request the full-sized image, thumbnail it, and return the smaller image to the requesting client.

I was actually pretty close last night, but sleepiness eventually got the better of me. This is a functioning version of last night's thumbnailer:
class ThumbNail
def initialize(app); @app = app end
def call(env)

if env['REQUEST_URI'] =~ /_sm.jpe?g/
env['REQUEST_URI'].sub!(/_sm\./, '.')
env['REQUEST_PATH'].sub!(/_sm\./, '.')
env['PATH_INFO'].sub!(/_sm\./, '.')
http_code, headers, original_body = @app.call(env)

img_data = ''
body.each do |data|
img_data << data
end


ImageScience.with_image_from_memory(img_data) do |img|
img.resize(100, 150) do |small|
small.save "/tmp/small.jpg"
end
end

f = File.open "/tmp/small.jpg"
[http_code, headers, f.read]
else
@app.call(env)
end
end
end

use ThumbNail
Missing last night was the bold section, where I actually use the body response from a Rack application like it is supposed to be used—with an each iteration.

I could delete the code and start on implementation proper, but first I would like to explore building a cache for the thumbnails. There is no sense in generating the images on each request. In this case, the call method needs to decide if a cache hit occurs and, if not, generate the thumbnail:
  # Main Rack call responder
def call(env)
if env['REQUEST_URI'] =~ /_sm.jpe?g/
# Handle thumbnail requests
unless File.exists?(filename(env))
mk_thumbnail(env)
end

small = File.new(filename(env)).read
[200, { }, small]
else
# Pass-thru non thumbnail requests
@app.call(env)
end
end
The rest of the code remains essentially the same. The complete, seemingly functional Rack handler:
class ThumbNail
# hard-code the store location (could be set via use options)
STORE = '/var/cache/thumbnail'

# Basic Rack initialization
def initialize(app); @app = app end

# Main Rack call responder
def call(env)
if env['REQUEST_URI'] =~ /_sm.jpe?g/
# Handle thumbnail requests
unless File.exists?(filename(env))
mk_thumbnail(env)
end

small = File.new(filename(env)).read
[200, { }, small]
else
# Pass-thru non thumbnail requests
@app.call(env)
end
end

# Make the thumbnail, storing it in the cache for subsequent
# requests
def mk_thumbnail(env)
$stderr.puts "[thumbnail] building thumbnail..."
cache_filename = filename(env)
path = cache_filename.sub(/\/[^\/]+$/, '')
FileUtils.mkdir_p(path)

env['REQUEST_URI'].sub!(/_sm\./, '.')
env['REQUEST_PATH'].sub!(/_sm\./, '.')
env['PATH_INFO'].sub!(/_sm\./, '.')

http_code, headers, body = @app.call(env)

img_data = ''
body.each do |data|
img_data << data
end

ImageScience.with_image_from_memory(img_data) do |img|
img.resize(200, 150) do |small|
small.save cache_filename
end
end
end

# Helper method for calculating the location of the file in the
# cache.
def filename(env)
"#{STORE}#{env['PATH_INFO']}"
end
end
Updating my Sinatra app to pull these images, I find the response time on my homepage much better (going from 1-2 seconds to ~100ms):



Tomorrow I delete all this code and then go back and do it right. Even in this limited spike, there is already code with which I am unhappy. A little BDD should clear that up.

Monday, October 5, 2009

Unsuccessful Image Science

‹prev | My Chain | next›

Heh. I really meant to replace the legacy site with the updated Sinatra / CouchDB site (currently in beta). But then I had one last look at the homepage. It was slow:



Instead of taking 1-2 seconds to download each meal photo as it does on the beta site, it takes ~100ms on the legacy site:



Image thumbnails.

To get thumbnails in my application stack, I think I'll try a little Rack trickery. First, though I need ImageScience:
jaynestown% sudo apt-get install libfreeimage3 libfreeimage-dev 
sudo gem install image_science
I try to write Rack middle ware that will translate a thumbnail request into a normal request, then use image science to resize the image. I try, but am ultimately unsuccessful. This is where I am forced to leave off for the night:
class ThumbNail
def initialize(app); @app = app end
def call(env)

if env['REQUEST_URI'] =~ /_sm.jpe?g/
env['REQUEST_URI'].sub!(/_sm\./, '.')
env['REQUEST_PATH'].sub!(/_sm\./, '.')
env['PATH_INFO'].sub!(/_sm\./, '.')
http_code, headers, original_body = @app.call(env)

ImageScience.with_image_from_memory(original_body) do |img|
img.resize(100, 150) do |small|
small.save "/tmp/small.jpg"
end
end

f = File.open "/tmp/small.jpg"
[http_code, headers, f.read]
else
@app.call(env)
end
end
end

use ThumbNail
I am getting this error in the Thin logs:
jaynestown% thin -R config.ru start
>> Thin web server (v1.2.2 codename I Find Your Lack of Sauce Disturbing)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:3000, CTRL+C to stop
!! Unexpected error while processing request: wrong argument type Rack::CommonLogger (expected String)
I will pick up tomorrow to continue working with ImageScience. If I am not able to make any more headway, I will drop down to RMagick.

Update: Ah, the original_body is not a string—it is something that responds to :each.

Sunday, October 4, 2009

Two More Features Deployed

‹prev | My Chain | next›

I have two new features to deploy to the beta site tonight: a poor man's CMS, which I am using for an "about us" page, and an ingredients index page. I have already smoke tested the about us page, so tonight it is time for the ingredients:



Ew. Not such a good start. That's not a 404—that is a RestClient resource not found. Specifically, the by-ingredients CouchDB view has not been uploaded to the development database. For that, I use a rake task wrapped around couch_docs:
jaynestown% rake couchdb:load_design_docs
(in /home/cstrom/repos/eee-code)
jaynestown%
With that, I have the ingredient index page:



A little bit of less CSS:
#index-of-ingredients {
td {
vertical-align: top;
}
.ingredient {
font-weight: bold;
}
p {
padding-left:10px;
text-indent:-10px;
}
}
And I have a decent looking ingredient index:



Last up, I add the new resources to the footer of the site:
  #footer
%div
%a{:href => '/'}= "Home"
|
%a{:href => '/feedback'}= "Contact"
|
%a{:href => '/recipes/search?q='}= "Recipe List"
|
%a{:href => '/ingredients'}= "Ingredient Index"
|
%a{:href => '/about'}= "About This Site"
With that, I redeploy:
jaynestown% rake vlad:stop_app                                         
jaynestown% rake vlad:update vlad:migrate vlad:start_app
I do not have quite every feature from the legacy site complete, but I am done enough. Tomorrow, I will point DNS to the new site, officially putting the site on Sinatra / CouchDB.

Saturday, October 3, 2009

Cucumber Driven Development with CouchDB

‹prev | My Chain | next›

Last in the ingredient index feature is ignoring ingredients that are too common to be worth including in the index. The Cucumber scenario describing this:
jaynestown% cucumber features/ingredient_index.feature:17                       
Sinatra::Test is deprecated; use Rack::Test instead.
Feature: Ingredient index for recipes

As a user curious about ingredients or recipes
I want to see a list of ingredients
So that I can see a sample of recipes in the cookbook using a particular ingredient

Scenario: Scores of recipes sharing an ingredient # features/ingredient_index.feature:17
Given 120 recipes with "butter" # features/ingredient_index.feature:19
When I visit the ingredients page # features/step_definitions/ingredient_index.rb:22
Then I should not see the "butter" ingredient # features/ingredient_index.feature:21

1 scenario (1 undefined)
3 steps (1 skipped, 2 undefined)
0m1.342s

You can implement step definitions for undefined steps with these snippets:

Given /^120 recipes with "([^\"]*)"$/ do |arg1|
pending
end

Then /^I should not see the "([^\"]*)" ingredient$/ do |arg1|
pending
end
The given-120-recipes step can be defined with a simple 120.times block:
Given /^(\d+) recipes with "([^\"]*)"$/ do |count, ingredient|
date_st = Date.new(2009, 10, 3)

count.to_i.times do |i|
date = date_st - i
permalink = date.to_s + "-recipe"

recipe = {
:title => "Yet another #{ingredient} recipe",
:type => 'Recipe',
:published => true,
:date => date,
:preparations => [{'ingredient' => {'name' => ingredient}}]
}

RestClient.put "#{@@db}/#{permalink}",
recipe.to_json,
:content_type => 'application/json'
end
end
The second step in that scenario, visiting the index of ingredients page, is the same as the step in the previous scenario. In other words, I do not have redefine that step. This leaves me the last step, asserting that this ingredient is not included in the index:
Then /^I should not see the "([^\"]*)" ingredient$/ do |ingredient|
response.should_not have_selector(".ingredient",
:content => ingredient)
end
That step fails, with lots, and lots of <a> tags:
jaynestown% cucumber features/ingredient_index.feature:17
Sinatra::Test is deprecated; use Rack::Test instead.
Feature: Ingredient index for recipes

As a user curious about ingredients or recipes
I want to see a list of ingredients
So that I can see a sample of recipes in the cookbook using a particular ingredient

Scenario: Scores of recipes sharing an ingredient # features/ingredient_index.feature:17
Given 120 recipes with "butter" # features/step_definitions/ingredient_index.rb:22
When I visit the ingredients page # features/step_definitions/ingredient_index.rb:43
Then I should not see the "butter" ingredient # features/step_definitions/ingredient_index.rb:64
expected following output to omit a <.ingredient>butter</.ingredient>:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
<title>EEE Cooks: Ingredient Index</title>
<link href="/stylesheets/style.css" rel="stylesheet" type="text/css">
</head>
<html><body>
<div id="header">
<div id="eee-header-logo">
<a href="/">
<img alt="Home" src="/images/eee_corner.png"></a>
</div>
</div>
<h1>
Ingredient Index
</h1>
<table><tr>
<td class="col1">
<p>
<span class="ingredient">
butter
</span>
<span class="recipes">
<a href="/recipes/2009-06-26-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-06-27-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-06-28-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-06-29-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-06-30-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-07-01-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-07-02-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-07-03-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-07-04-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-07-05-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-07-26-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-07-27-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-07-28-recipe">Yet another butter recipe</a>, <a href="/recipes/2009-07-29-recipe">Yet another butter recipe</a>,
...
</body></html>
</html>
(Spec::Expectations::ExpectationNotMetError)
features/ingredient_index.feature:21:in `Then I should not see the "butter" ingredient'

Failing Scenarios:
cucumber features/ingredient_index.feature:17 # Scenario: Scores of recipes sharing an ingredient

1 scenario (1 failed)
3 steps (1 failed, 2 passed)
0m1.734s
To get that passing I need to drop down into the CouchDB view. Specifically, when "reducing", I need to check the arrays of recipes being associated with the ingredient. If there are more than 100 recipes using said ingredient, I need to return an empty list:
function(keys, values, rereduce) {
if (rereduce) {
var ret = [];
for (var i=0; i<values.length; i++) {
ret = ret.concat(values[i]);
}
return ret.length > 100 ? [] : ret;
}
else {
return values;
}
}
It was noted the other day that my "reduce" here is not really reducing. In this case it works, but at some point I would hit a performance wall (at least according to the documentation). If I were looking to to avoid that wall, I might do so here—possibly even getting the reduce down to the O(logn) suggested by the documentation. For now, I do not need it—performance of the view is perfectly fine with ~500 recipes and ~5000 ingredients—so I will stick with what I have.

Before the scenario will pass, I have to update the Haml template to ignore ingredients with no recipes:
...
- @ingredients.each_slice((@ingredients.size.to_f/2).round) do |batch|
- col = col + 1
%td{:class => "col#{col}"}
- batch.reject{|i| i['value'].empty?}.each do |ingredient|
%p
...
All of my unit (RSpec) tests still pass. And now the scenario does as well:
jaynestown% cucumber features/ingredient_index.feature:17
Sinatra::Test is deprecated; use Rack::Test instead.
Feature: Ingredient index for recipes

As a user curious about ingredients or recipes
I want to see a list of ingredients
So that I can see a sample of recipes in the cookbook using a particular ingredient

Scenario: Scores of recipes sharing an ingredient # features/ingredient_index.feature:17
Given 120 recipes with "butter" # features/step_definitions/ingredient_index.rb:22
When I visit the ingredients page # features/step_definitions/ingredient_index.rb:43
Then I should not see the "butter" ingredient # features/step_definitions/ingredient_index.rb:64

1 scenario (1 passed)
3 steps (3 passed)
0m1.699s
Tomorrow I will deploy my latest features before deciding where to move onto next.

Friday, October 2, 2009

Inside Out Ingredients

‹prev | My Chain | next›

I should be done with the ingredient index page. I have a CouchDB view set up to pull this information back from the server. I have the Sinatra resource pulling from said CouchDB view. I even have the Haml view displaying the data.

I should be done, but I need verify that the pieces fit together, which is what Cucumber is for. The scenario from which all this started now stands at:
jaynestown% cucumber features/ingredient_index.feature:7 -s                     
Sinatra::Test is deprecated; use Rack::Test instead.
Feature: Ingredient index for recipes

As a user curious about ingredients or recipes
I want to see a list of ingredients
So that I can see a sample of recipes in the cookbook using a particular ingredient

Scenario: A couple of recipes sharing an ingredient
Given a "Cookie" recipe with "butter" and "chocolate chips"
And a "Pancake" recipe with "flour" and "chocolate chips"
When I visit the ingredients page
Then I should see the "chocolate chips" ingredient
And "chocolate chips" recipes should include "Cookie" and "Pancake"
And I should see the "flour" ingredient
And "flour" recipes should include only "Pancake"

1 scenario (1 undefined)
7 steps (4 undefined, 3 passed)
0m2.190s

You can implement step definitions for undefined steps with these snippets:

Then /^I should see the "([^\"]*)" ingredient$/ do |arg1|
pending
end

Then /^"([^\"]*)" recipes should include "([^\"]*)" and "([^\"]*)"$/ do |arg1, arg2, arg3|
pending
end

Then /^"([^\"]*)" recipes should include only "([^\"]*)"$/ do |arg1, arg2|
pending
end
I define the first step as:
Then /^I should see the "([^\"]*)" ingredient$/ do |ingredient|
response.should have_selector(".ingredient",
:content => ingredient)
end
Running the scenario, I find:
jaynestown% cucumber features/ingredient_index.feature:7 -s
Sinatra::Test is deprecated; use Rack::Test instead.
Feature: Ingredient index for recipes

As a user curious about ingredients or recipes
I want to see a list of ingredients
So that I can see a sample of recipes in the cookbook using a particular ingredient

Scenario: A couple of recipes sharing an ingredient
Given a "Cookie" recipe with "butter" and "chocolate chips"
And a "Pancake" recipe with "flour" and "chocolate chips"
When I visit the ingredients page
Then I should see the "chocolate chips" ingredient
expected following output to contain a <.ingredient>chocolate chips</.ingredient> tag:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
<title>EEE Cooks: Ingredient Index</title>
<link href="/stylesheets/style.css" rel="stylesheet" type="text/css">
</head>
<html><body>
<div id="header">
<div id="eee-header-logo">
<a href="/">
<img alt="Home" src="/images/eee_corner.png"></a>
</div>
</div>
<h1>
Ingredient Index
</h1>
<table><tr>
<td class="col1">
<p>
<span class="ingredient">
value
</span>
<span class="recipes">
<a href="/recipes/"></a>
</span>
</p>
...
Ew. An ingredient of "value"? A quick investigation identifies a discrepancy between that the view expects and what CouchDB returns (missing the "keys" attribute in the specification).

I could have just as easily viewed the page in a browser to see this error. The benefit of using Cucumber to find this error, of course, is that I never have to manually find this bug in a browser again. I have a high degree of confidence that I have an integration test that is valuable (i.e. it actually found a bug).

After fixing that, I define the next missing step as:
Then /^"([^\"]*)" recipes should include "([^\"]*)" and "([^\"]*)"$/ do |ingredient, arg2, arg3|
response.should have_selector(".recipes") do |span|
span.should have_selector("a", :content => arg2)
span.should have_selector("a", :content => arg3)
end
end
The response ought to have an element with a class="recipes". That element should have child <a> elements, which should be linking the recipe titles from the feature ("chocolate chips" recipes should include "Cookie" and "Pancake").

Last up, I need a definition that fits:
Then "flour" recipes should include only "Pancake"
For this, I need to break out the XPath. I am looking to verify that a <p> tag contains a span with the "flour" ingredient and that also contains another span that has the recipe title somewhere in it. The step definition that verifies this:
Then /^"([^\"]*)" recipes should include only "([^\"]*)"$/ do |ingredient, recipe|
response.should have_xpath("//p[contains(span, '#{ingredient}')]/span[contains(., '#{recipe}')]")
end
For good measure, I would like to verify that there is only one <a> tag associated with that ingredient, so I add a second XPath expression:
Then /^"([^\"]*)" recipes should include only "([^\"]*)"$/ do |ingredient, recipe|
response.should have_xpath("//p[contains(span, '#{ingredient}')]/span[contains(., '#{recipe}')]")
response.should have_xpath("//p[contains(span, '#{ingredient}')]/span[count(a)=1]")
end
Just like that, I have a passing scenario:
jaynestown% cucumber features/ingredient_index.feature:7 -s
Sinatra::Test is deprecated; use Rack::Test instead.
Feature: Ingredient index for recipes

As a user curious about ingredients or recipes
I want to see a list of ingredients
So that I can see a sample of recipes in the cookbook using a particular ingredient

Scenario: A couple of recipes sharing an ingredient
Given a "Cookie" recipe with "butter" and "chocolate chips"
And a "Pancake" recipe with "flour" and "chocolate chips"
When I visit the ingredients page
Then I should see the "chocolate chips" ingredient
And "chocolate chips" recipes should include "Cookie" and "Pancake"
And I should see the "flour" ingredient
And "flour" recipes should include only "Pancake"

1 scenario (1 passed)
7 steps (7 passed)
0m0.665s
I have one more scenario describing the case in which "common" ingredients are excluded from the index. No one really wants to scan through an index and be confronted with 200+ recipes with salt in them. I will pick up with that scenario tomorrow.

Thursday, October 1, 2009

Ingredients, in Batches

‹prev | My Chain | next›

Picking back up with my ingredient index:
  it "should link to the recipes" do
render("/views/ingredients.haml")
response.should have_selector("a", :count => 3)
end
The expectation that there should be three links in the output comes from the setup block. There are two recipes, one of which appears under two different ingredients in the index:
  before(:each) do
@ingredients = [{'butter' =>
[
['recipe-id-1', 'title 1'],
['recipe-id-2', 'title 2']
]
},
{'sugar' =>
[
['recipe-id-2', 'title 2']
]
}]
end
When I run the spec, this latest example fails:
jaynestown% spec ./spec/views/ingredients.haml_spec.rb -cfs

ingredients.haml
- should have a list of ingredients
- should have a list of recipes using the ingredients
- should link to the recipes (FAILED - 1)

1)
'ingredients.haml should link to the recipes' FAILED
expected following output to contain a <a/> tag:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html><body>
<p>
<span class="ingredient">
butter
</span>
<span class="recipes">
title 1, title 2
</span>
</p>
<p>
<span class="ingredient">
sugar
</span>
<span class="recipes">
title 2
</span>
</p>
butterrecipe-id-1title 1recipe-id-2title 2sugarrecipe-id-2title 2
</body></html>

./spec/views/ingredients.haml_spec.rb:30:

Finished in 0.03 seconds

3 examples, 1 failure
To get the 3 links (and make the example pass), I add links to the Haml template:
= @ingredients.each do |ingredient|
%p
%span.ingredient
= ingredient.keys.first
%span.recipes
= ingredient.values.first.map{|recipe| %Q|<a href="">#{recipe[1]}</a>|}.join(", ")
Easy enough, but the href attribute is empty. To drive that, I will focus on one recipe, one context:
  context "an ingredient is used in only one recipe" do
it "should have one link to that recipe" do
render("/views/ingredients.haml")
response.should have_selector("a",
:href => "/recipes/recipe-id-1",
:content => "title 1",
:count => 1)
end
end
For good measure, I also describe the case of multiple recipes sharing a recipe:
  context "an ingredient is used in multiple recipes" do
it "should have multiple links to that recipe" do
render("/views/ingredients.haml")
response.should have_selector("a",
:href => "/recipes/recipe-id-2",
:content => "title 2",
:count => 2)
end
end
Adding the hyperlink to the <a> in the Haml templaate:
      = ingredient.values.first.map{|recipe| %Q|<a href="/recipes/#{recipe[0]}">#{recipe[1]}</a>|}.join(", ")
Makes all examples pass:
jaynestown% spec ./spec/views/ingredients.haml_spec.rb -cfs

ingredients.haml
- should have a list of ingredients
- should have a list of recipes using the ingredients
- should link to the recipes

ingredients.haml an ingredient is used in only one recipe
- should have one link to that recipe

ingredients.haml an ingredient is used in multiple recipes
- should have one link to that recipe

Finished in 0.02 seconds

5 examples, 0 failures
The last thing that needs to go on this page are two columns. In the legacy site, we found that the index looked best with two columns:



I write two, paired examples for this output behavior:
  it "should put half of the ingredients in the first column" do
render("/views/ingredients.haml")
response.should have_selector(".col1", :content => "butter")
end

it "should put the other half of the ingredients in the second column" do
render("/views/ingredients.haml")
response.should have_selector(".col2", :content => "sugar")
end
I can make that pass with a table (for the two columns) and batching the ingredients into two:
%table
%tr
- col = 0
- @ingredients.each_slice((@ingredients.size.to_f/2).round) do |batch|
- col = col + 1
%td{:class => "col#{col}"}
= batch.each do |ingredient|
...
I dislike using accumulators (col), but there is no each_slice_with_index in Enumerator, so...

With that, I am done with my inside code work on this mini-feature. Tomorrow, I will work my way back out to the Cucumber scenario to verify that all the pieces work when put together.