Reduce Numbers, Not Sizes

‹prev | My Chain | next›

Picking up from last night I am currently unable to get even the simplest of once rock-solid Cucumber scenarios to pass:

cstrom@cstrom-laptop:~/repos/eee-code$ cucumber features/browse_meals.feature:7
Feature: Browse Meals

  So that I can find meals made on special occasions
  As a person interested in exploring meals and how they drive certain recipes
  I want to browse meals by date

  Scenario: Browsing a meal in a given year                                          # features/browse_meals.feature:7
    Given a "Even Fried, They Won't Eat It" meal enjoyed in 2009                     # features/step_definitions/meal_details.rb:1
    And a "Salad. Mmmm." meal enjoyed in 2008                                        # features/step_definitions/meal_details.rb:1
    When I view the list of meals prepared in 2009                                   # features/step_definitions/meal_details.rb:44
      HTTP status code 500 (RestClient::RequestFailed)
      /usr/lib/ruby/1.8/net/http.rb:543:in `start'
      ./features/support/../../eee.rb:114:in `GET (?-mix:\/meals\/(\d+))'
      (eval):2:in `visit'
      ./features/step_definitions/meal_details.rb:45:in `/^I view the list of meals prepared in 2009$/'
      features/browse_meals.feature:11:in `When I view the list of meals prepared in 2009'
    Then the "Even Fried, They Won't Eat It" meal should be included in the list     # features/step_definitions/meal_details.rb:75
    And the "Salad. Mmmm." meal should not be included in the list                   # features/step_definitions/meal_details.rb:79
    When I follow the link to the list of meals in 2008                              # features/step_definitions/meal_details.rb:59
    Then the "Even Fried, They Won't Eat It" meal should not be included in the list # features/step_definitions/meal_details.rb:79
    And the "Salad. Mmmm." meal should be included in the list                       # features/step_definitions/meal_details.rb:75

Failing Scenarios:
cucumber features/browse_meals.feature:7 # Scenario: Browsing a meal in a given year

1 scenario (1 failed)
8 steps (1 failed, 5 skipped, 2 passed)
0m0.907s

Alas! This is my laziness (not the good kind) catching up with me. When I was learning CouchDB, I failed to heed an important CouchDB idiom: reduce to a single value when map-reducing. In the heady days of CouchDB 0.9, one could get away with reducing to smaller lists of values. Those same map-reduces now result in wonderful Erlang errors like:

[Wed, 03 Feb 2010 03:00:07 GMT] [error] [<0.4416.0>] {error_report,<0.24.0>,
    {<0.4416.0>,crash_report,
     [[{initial_call,{couch_view_group,init,['Argument__1']}},
       {pid,<0.4416.0>},
       {registered_name,[]},
       {error_info,
           {exit,
               {reduce_overflow_error,
                   <<"Reduce output must shrink more rapidly: Current output: '[[{\"_id\": \"2008-05-13\",\"_rev\": \"1-3c7fe582a51492dc6a756e0
721373165\",\"type\": \"Meal\",\"title\": \"Salad. '... (first 100 of 476 bytes)">>},
               [{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
       {ancestors,
           [couch_view,couch_secondary_services,couch_server_sup,<0.1.0>]},
       {messages,[]},
       {links,[<0.4418.0>,<0.61.0>]},
       {dictionary,[]},
       {trap_exit,true},
       {status,running},
       {heap_size,1597},
       {stack_size,24},
       {reductions,675}],
...

What this is telling me is that the reduce in map-reduce does not refer to reducing the size of values, rather it refers to the total number of values (which should be very close to one). The "Current output" in the backtrace is a smaller version of each meal document (just the title and date of each). The output is smaller, but not reduced.

At least two of my "reduce" functions were plain silly and need to be deleted. I have a by-year map, which I then reduce with this nonsensical function:

function(keys, values, rereduce) { return values; }

There is no reduce going on there at all! I added that before I realized that you can select ranges in a map view (non-reduced) thusly:

  url = "#{@@db}/_design/meals/_view/by_date?" +
    "startkey=%22#{year}-00-00%22&" +
    "endkey=%22#{year}-99-99%22"
  data = RestClient.get url

Removing two silly reduces gets me past Erlang backtraces and on to the following:

 Scenario: Browsing a meal in a given year                                          # features/browse_meals.feature:7
    Given a "Even Fried, They Won't Eat It" meal enjoyed in 2009                     # features/step_definitions/meal_details.rb:1
    And a "Salad. Mmmm." meal enjoyed in 2008                                        # features/step_definitions/meal_details.rb:1
    When I view the list of meals prepared in 2009                                   # features/step_definitions/meal_details.rb:44
      undefined local variable or method `response' for #<MyWorld:0x..fdb7afd40> (NameError)
      ./features/step_definitions/meal_details.rb:46:in `/^I view the list of meals prepared in 2009$/'
      features/browse_meals.feature:11:in `When I view the list of meals prepared in 2009'

Ugh. This is due to my shift to rack test last night. The response method is no longer available—I have to use one of response_body or response_code.

After changing that in nearly ever Cucumber step definition that I have, I have all of my scenarios passing except for searching (I have not installed couchdb-lucene yet) and a by-ingredients index page that relies on a problematic map-reduce. I will pick up tomorrow addressing those.

Day #2