Tuesday, July 14, 2009

Over 1,000 Documents

‹prev | My Chain | next›

With the couch_design_docs gem, I am able to easily drop and recreate my development (and ultimately my production) CouchDB database. This all began because I wanted to load all of my meals and recipes in to my development database. So let's do it!

First up, I drop my old database:



Then recreate it:



Now I reload all of my design documents with couch_design_docs and the rake task that I created yesterday:
require 'couch_design_docs'
task :load_design_docs do
CouchDesignDocs.upload_dir("http://localhost:5984/eee", "couch/_design")
end
After running rake load_design_docs, I do a quick double check to find that yes, my design documents are now in the database:



Now that I think about it, I will likely be dropping and recreating my development CouchDB quite a bit now that the couch_design_docs gem makes it easy to reload my design docs. Rather than going through that manual process each time, I ought to automate this with rake (and do a little namespace organization while I am at it):
namespace :couchdb do

desc "Drop and re-create the CouchDB database, loading the design documents after creation"
task :reset => [:drop, :create, :load_design_docs]

desc "Create a new version of the CouchDB database"
task :create do
RestClient.put DB, { }
end

desc "Delete the current the CouchDB database"
task :drop do
RestClient.delete DB
end

require 'couch_design_docs'

desc "Load (replacing any existing) all design documents"
task :load_design_docs do
CouchDesignDocs.upload_dir(DB, "couch/_design")
end
end
It has been a while since I uploaded documents from my legacy Rails application into my new CouchDB store. The instructions for uploading my recipes include re-opening the Recipe class to add methods for CouchDB JSON generation:
class Recipe < ActiveRecord::Base

def _id; date.to_s + '-' + self.label end

def tag_names; tags.map(&:name) end

def _attachments
if self.image && self.image.filename
{
self.image.filename =>
{
:data => Base64.encode64(File.open(self.image.full_filename).read).gsub(/\n/, ''),
:content_type => "image/jpeg"
}
}
end
end

def couch_json
def self.type; "Recipe" end

self.to_json( :methods => [:tag_names, :_id, :_attachments],
:include => {
:preparations =>
{
:include =>
{
:ingredient => { :except => :id }
},
:except => [:ingredient_id, :recipe_id, :id]
},
:tools => { :except => [:id, :label] } },
:except => [:id, :label]).sub(/\{/, '{"type":"Recipe",')
end
end
I then run through each recipe in my legacy rails database, using the couch_json method to upload these documents to CouchDB:
>> ActiveSupport.use_standard_json_time_format=true
>> require 'restclient'
>> Recipe #load the current recipe class so that I re-open, not define
>> class Recipe < ActiveRecord::Base ... # the rest of the re-opened class from above
>> Recipe.all.each {|r| RestClient.put "http://localhost:5984/eee/#{r._id}", r.couch_json, :content_type => 'application/json'}
Similarly, the instructions for uploading my meals also have me re-open the Meal class to add methods for CouchDB JSON:
class Meal < ActiveRecord::Base
# IDs - date suffices for a meal (we never do more than one meal per day)
def _id; date.to_s end

# JSON for the CouchDB Document
def couch_json
self.to_json(:methods => [:_id, :menu, :type, :_attachments], :except => [:id, :image_old])
end

# For uploading meal images
def _attachments
if self.image && self.image.filename
{
self.image.filename =>
{
:data => Base64.encode64(File.open(self.image.full_filename).read).gsub(/\n/, ''),
:content_type => "image/jpeg"
}
}
end
end

# For the menu
def menu; menu_items.map(&:name) end

# To distinguish between meals, recipes, etc.
def type; self.class.to_s end
end
And, in Irb:
>> Meal.all.each{|m| RestClient.put "http://localhost:5984/eee/#{m._id}", m.couch_json, :content_type => 'application/json'}
With data in the application, it is time to fire up Sinatra. Initially, I get an error about missing the "kids" document in CouchDB (used to lookup nicknames that we use for the kids). After creating an empty kids document, I am able to navigate to http://localhost:4567 to see:



Yay! Over 1,000 documents (recipes and meals) are now loaded in my app and it seems to be working. I will do some smoke testing tomorrow to make sure that all is well. Then I need to figure out how to handle seed data like that "kids" document.

Update: The instructions were updated to reflect the need to format the dates in ISO 8601 format by setting ActiveSupport.use_standard_json_time_format=true.

Monday, July 13, 2009

CouchDesignDocs Gem Ready for Real-World Use

‹prev | My Chain | next›

Before the couch_design_docs gem is ready for prime-time (i.e. ready to be used in my Sinatra / CouchDB app), I want a simpler API than is currently exposed. Currently I need to instantiate both a Store object (to describe the target CouchDB store) and a Directory object (to describe the design documents stored on the file system). The Store object then needs to upload a hash representation of the Directory object. I would much prefer a single method to upload a design documents directory to a CouchDB store. Something like:
CouchDesignDocs.upload_dir(URI, DIR)
This convenience method should instantiate Store and Directory objects just I have to by-hand now. The instantiated Store object should receive the :load message with an argument of the hash from Directory. As an RSpec example:
describe CouchDesignDocs do
it "should be able to load directory/JS files into CouchDB as design docs" do
store = mock("Store")
Store.stub!(:new).and_return(store)

dir = mock("Directory")
dir.stub!(:to_hash).and_return({ "foo" => "bar" })
Directory.stub!(:new).and_return(dir)

store.
should_receive(:load).
with({ "foo" => "bar" })

CouchDesignDocs.upload_dir("uri", "fixtures")
end
end
The example fails with
cstrom@jaynestown:~/repos/couch_design_docs$ spec spec/couch_design_docs_spec.rb
F...........

1)
NoMethodError in 'CouchDesignDocs should be able to load directory/JS files into CouchDB as design docs'
undefined method `upload_dir' for CouchDesignDocs:Module
./spec/couch_design_docs_spec.rb:16:

Finished in 0.011794 seconds

12 examples, 1 failure
I change the message by defining the method. I change the next message ("wrong number of arguments (2 for 0)") by defining the method with two arguments:
  def self.upload_dir(db_uri, dir)
end
With that I get this failure:
cstrom@jaynestown:~/repos/couch_design_docs$ spec spec/couch_design_docs_spec.rb
F...........

1)
Spec::Mocks::MockExpectationError in 'CouchDesignDocs should be able to load directory/JS files into CouchDB as design docs'
Mock 'Store' expected :load with ({"foo"=>"bar"}) once, but received it 0 times
./spec/couch_design_docs_spec.rb:12:

Finished in 0.012137 seconds

12 examples, 1 failure
I make that example pass by defining the methods as I planned from the beginning:
  def self.upload_dir(db_uri, dir)
store = Store.new(db_uri)
dir = Directory.new(dir)
store.load(dir.to_hash)
end
After updating the README and the History.txt file, I create an updated gemspec with the built-in Bones rake command:
rake gem:spec
Then I upload to github.
(commit)

Finally, I replace the code from which the couch_design_docs gem was extracted with a call to the gem:
Before do
# For mocking & stubbing in Cucumber
$rspec_mocks ||= Spec::Mocks::Space.new

# Create the DB
RestClient.put @@db, { }

# Upload the design documents with a super-easy gem :)
CouchDesignDocs.upload_dir(@@db, 'couch/_design')

end
Nice! At one point, that Before block was well over a hundred lines long, mostly because the design documents were embedded directly in that block. Now it is a single line.

To make sure that everything is still working, I break the design docs out into the component .js files:
cstrom@jaynestown:~/repos/eee-code$ find couch/
couch/
couch/_design
couch/_design/lucene
couch/_design/lucene/transform.js
couch/_design/recipes
couch/_design/recipes/views
couch/_design/recipes/views/by_date
couch/_design/recipes/views/by_date/map.js
couch/_design/meals
couch/_design/meals/views
couch/_design/meals/views/by_month
couch/_design/meals/views/by_month/map.js
couch/_design/meals/views/by_month/reduce.js
couch/_design/meals/views/by_date
couch/_design/meals/views/by_date/map.js
couch/_design/meals/views/count_by_month
couch/_design/meals/views/count_by_month/map.js
couch/_design/meals/views/count_by_month/reduce.js
couch/_design/meals/views/count_by_year
couch/_design/meals/views/count_by_year/map.js
couch/_design/meals/views/count_by_year/reduce.js
couch/_design/meals/views/by_year
couch/_design/meals/views/by_year/map.js
couch/_design/meals/views/by_year/reduce.js
Running the various Cucumber scenarios that exercise the full-stack, including the CouchDB views/design documents as well as the couchdb-lucene design document, I find no failures:
27 scenarios
9 skipped steps
1 undefined step
226 passed steps
Say, that's a really convenient gem, it'd be nice to have easy access to it from the command line! I add it to my Rakefile thusly:
task :load_design_docs do
CouchDesignDocs.upload_dir("http://localhost:5984/eee", "couch/_design")
end
That winds up work on the couch_design_docs gem—an excursion made quite enjoyable thanks to Bones. I still have some features that I would like to add (ability to unit test the .js files, Javascript function re-use via Erb includes), but it serves my purposes really well as-is.

Up tomorrow: figuring out where I was before I decided extracting a gem might be a good idea.

Sunday, July 12, 2009

When I Say put, I Mean put!

‹prev | My Chain | next›

As of last night, the CouchDB Store class in the couch_design_docs gem is capable of putting new documents in a CouchDB database. That functionality is sufficient for working with my testing database, where I drop and recreate the entire database with each testing run. I discovered that this is not as helpful in real-world usage when I tried to load design documents into my existing development database. So it is back into the code to add this functionality...

In RSpec parlance, the Store class should be able to put a new document. The Store.put class method is already able to put documents. What I need now is a possible destructive put or, in idiomatic ruby, Store.put!. The example that describes using this method for new documents:
    it "should be able to put a new document" do
Store.
should_receive(:put).
with("uri", { })

Store.put!("uri", { })
end
Thus starts the change-the-message or make-it-pass BDD cycle. First up, change the message in this:
cstrom@jaynestown:~/repos/couch_design_docs$ spec spec/couch_design_docs_spec.rb
.F.......

1)
NoMethodError in 'CouchDesignDocs::Store a valid store should be able to put a new document'
undefined method `put!' for CouchDesignDocs::Store:Class
./spec/couch_design_docs_spec.rb:30:

Finished in 0.010079 seconds

9 examples, 1 failure
To change that message, I add a put! method to the Store class:
    def self.put!
end
Another message to be changed:
cstrom@jaynestown:~/repos/couch_design_docs$ spec spec/couch_design_docs_spec.rb
.F.......

1)
ArgumentError in 'CouchDesignDocs::Store a valid store should be able to put a new document'
wrong number of arguments (2 for 0)
./spec/couch_design_docs_spec.rb:30:in `put!'
./spec/couch_design_docs_spec.rb:30:

Finished in 0.010416 seconds

9 examples, 1 failure
As the example describes, I need to be able to pass two arguments to this method, the URL of the document and the document itself:
    def self.put!(path, doc)
end
And now, I receive this failure:
cstrom@jaynestown:~/repos/couch_design_docs$ spec spec/couch_design_docs_spec.rb
.F.......

1)
Spec::Mocks::MockExpectationError in 'CouchDesignDocs::Store a valid store should be able to put a new document'
expected :put with ("uri", {}) once, but received it 0 times
./spec/couch_design_docs_spec.rb:26:

Finished in 0.009909 seconds

9 examples, 1 failure
At this point, I no longer need to change the message, but am ready to make the example pass:
    def self.put!(path, doc)
self.put(path, doc)
end
So far, there is no difference between put and put!. Let's make a difference. If the put fails, it should delete so that a subsequent put will succeed. Or, in RSpec form:
    it "should delete existing docs if first put fails" do
Store.
stub!(:put).
and_raise(RestClient::RequestFailed)

Store.should_receive(:delete)

Store.put!("uri", { })
end
To get that passing, I add a rescue block that deletes the existing document:
    def self.put!(path, doc)
self.put(path, doc)
rescue RestClient::RequestFailed
self.delete

end
Now I need to add another put call after the delete in the rescue block. This is a little tricky to describe with RSpec. Something like this does not work:
    it "should retry the put if the first fails" do
Store.
should_receive(:put).
exactly(:twice).
and_raise(RestClient::RequestFailed)

Store.stub!(:delete)

Store.put!("uri", { })
end
I try to implement this with a second put:
    def self.put!(path, doc)
self.put(path, doc)
rescue RestClient::RequestFailed
self.delete(path)
self.put(path, doc)
end
The problem with this is that both put calls now raise errors. The first is caught by the rescue block, but the second one is uncaught:
cstrom@jaynestown:~/repos/couch_design_docs$ spec spec/couch_design_docs_spec.rb
..FF.......

1)
RestClient::RequestFailed in 'CouchDesignDocs::Store a valid store should delete existing docs if first put fails'
HTTP status code
/home/cstrom/repos/couch_design_docs/lib/couch_design_docs/store.rb:24:in `put!'
./spec/couch_design_docs_spec.rb:42:

2)
RestClient::RequestFailed in 'CouchDesignDocs::Store a valid store should retry the put if the first fails'
HTTP status code
/home/cstrom/repos/couch_design_docs/lib/couch_design_docs/store.rb:24:in `put!'
./spec/couch_design_docs_spec.rb:53:

Finished in 0.010869 seconds

11 examples, 2 failures
It is tempting to look upon this as a limitation of RSpec. Maybe it is, but I take this as an opportunity to create a delete_and_put method:
    it "should be able to delete and put" do
Store.
should_receive(:delete).
with("uri", { })

Store.
should_receive(:put).
with("uri", { })

Store.delete_and_put("uri", { })
end
Making that example pass is a simple matter of moving the delete and put calls in the rescue block into a new delete_and_put method:
    def self.delete_and_put(path, doc)
self.delete(path)
self.put(path, doc)
end
I can then redo the earlier, should delete upon put failure example, to read:
    it "should delete existing docs if first put fails" do
Store.
stub!(:put).
and_raise(RestClient::RequestFailed)

Store.
should_receive(:delete_and_put).
with("uri", { })

Store.put!("uri", { })
end
The final implementation of the put! method then becomes:
    def self.put!(path, doc)
self.put(path, doc)
rescue RestClient::RequestFailed
self.delete_and_put(path, doc)
end
Nice! As I said, it was tempting to think of the difficulties with the two and_raise calls as a limitation of RSpec, but that "limitation" led a cleaner implementation.

After switching the Store.load code to use the new put! method, I give the gem another try on my development database:
cstrom@jaynestown:~/repos/eee-code$ irb
>> require 'couch_design_docs'
=> true
>> dir = CouchDesignDocs::Directory.new("/home/cstrom/repos/eee-code/couch/_design")
=> #<CouchDesignDocs::Directory:0xb79776bc @couch_view_dir="/home/cstrom/repos/eee-code/couch/_design">
>> store = CouchDesignDocs::Store.new("http://localhost:5984/eee")
=> #<CouchDesignDocs::Store:0xb797126c @url="http://localhost:5984/eee">
>> store.load(dir.to_hash)
=> {"lucene"=>{"transform"=>"function(doc) { … }"}}
Nice! The store.load call failed yesterday because of the put instead of the new put!.

Before calling it a day, I update the version number of my gem to 1.0.1 (I probably should have started below 1.0) by editing the lib/couch_design_docs.rb file created for me by Bones to include:
  VERSION = '1.0.1'
Then I regenerate my gemspec:
cstrom@jaynestown:~/repos/couch_design_docs$ rake gem:spec
(in /home/cstrom/repos/couch_design_docs)
(commit)

Up tomorrow: a convenience method so that I can upload design docs with a single call, and then I will replace the current code doing this in my application with said convenience method.

Saturday, July 11, 2009

Working with the New Gem

‹prev | My Chain | next›

I continue working on couch_design_docs, a gem to load javascript files from the file system into my CouchDB store as design docs.

Creating a gemspec (needed by github) is easy with bones:
cstrom@jaynestown:~/repos/couch_design_docs$ rake gem:spec
(in /home/cstrom/repos/couch_design_docs)
cstrom@jaynestown:~/repos/couch_design_docs$ git status
# On branch master
# Untracked files:
# (use "git add ..." to include in what will be committed)
#
# couch_design_docs.gemspec
After pushing the changes to github and a little wait, I can now install my own gem:
cstrom@jaynestown:~$ gem install eee-c-couch_design_docs
WARNING: Installing to ~/.gem since /usr/lib/ruby/gems/1.8 and
/usr/bin aren't both writable.
Successfully installed eee-c-couch_design_docs-1.0.0
1 gem installed
I could also have installed from my local copy using bones's rake gem:install.

With the gem installed, I can take it for a ride:
cstrom@jaynestown:~/repos/eee-code$ irb
>> require 'couch_design_docs'
=> true
Cool! That was easy enough. To instantiate a Store object:
>> store = CouchDesignDocs::Store.new("http://localhost:5984/eee")
=> #<CouchDesignDocs::Store:0xb7a07c58 @url="http://localhost:5984/eee">
And a local document store:
>> dir = CouchDesignDocs::Directory.new("/home/cstrom/repos/eee-code/couch/_design")
=> #<CouchDesignDocs::Directory:0xb7970948 @couch_view_dir="/home/cstrom/repos/eee-code/couch/_design">
>> dir.to_hash
=> {"lucene"=>{"transform"=>"function(doc) { … }"}}
Nice! So, can I get the store to upload the design document?
>> store.load(dir.to_hash)
RestClient::RequestFailed: HTTP status code 409
from /usr/lib/ruby/gems/1.8/gems/rest-client-1.0/lib/restclient/request.rb:193:in `process_result'
from /usr/lib/ruby/gems/1.8/gems/rest-client-1.0/lib/restclient/request.rb:123:in `transmit'
from /usr/lib/ruby/1.8/net/http.rb:543:in `start'
Conflict, bummer. That is something missing from the gem—the ability to replace existing documents. For now, I will manually delete the couchdb-lucene design document:



With that, I can retry:
>> store.load(dir.to_hash)
=> {"lucene"=>{"transform"=>"function(doc) { … }" }}
Yay! The .js file from the filesystem was actually PUT in the CouchDB data store as desired.

I spend a little more time with the document store class—driving by example the ability to replace an existing design document. I hope to be able to finish up the gem tomorrow by adding some convenience methods and the ability to replace existing documents.
(commit)

Friday, July 10, 2009

A CouchDB Store PUTter

‹prev | My Chain | next›

Let's see, where was I?:
cstrom@jaynestown:~/repos/couch_design_docs$ spec ./spec/
couch_design_docs_spec.rb spec_helper.rb
cstrom@jaynestown:~/repos/couch_design_docs$ spec ./spec/couch_design_docs_spec.rb
...*

Pending:

CouchDesignDocs::Directory a valid directory should assemble all documents into a single docs structure
(you can do a better job with deep hash merging than that)
./spec/couch_design_docs_spec.rb:39

Finished in 0.006632 seconds

4 examples, 0 failures, 1 pending
Yeah, yeah. Very funny yesterday self. Still, I do need to get a little better.

I am converting this directory structure into a hash:
fixtures/a/b/c.js
fixtures/a/b/d.js
The two .js files generate these two hashes that need to be merged:
# fixtures/a/b/c.js =>
{
'a' => {
'b' => {
'c' => 'function(doc) { return true; }'
}
}
}

# fixtures/a/b/d.js =>
{
'a' => {
'b' => {
'd' => 'function(doc) { return true; }'
}
}
}
I have all of this working. The conversion of the directory / file structure to hash works well. The merging of the two hashes, not so well. I am using this recursive method to merge things:
   def deep_hash_merge(h1, h2)
h2.each_key do |k|
if h1.key? k
deep_hash_merge(h1[k], h2[k])
else
h1[k] = h2[k]
end
end
h1
end
Mostly, I do not like the line h1[k] = h2[k]—updating the original hash is an unnecessary side-effect. After noodling through the problem on my own I Google a bit to find how Rails has solved this. I use a scaled down version of this:
class Hash
def deep_merge(other)
self.merge(other) do |key, oldval, newval|
oldval.deep_merge(newval)
end
end
end
The new Hash#deep_merge method makes use of the optional block parameter for the Ruby core Hash#merge method. The block is evaluated only when the two source hashes have conflicting keys—exactly what I need. Specifically, I need to keep working down the two Hash trees (i.e. when merging both 'a' keys), until no conflicts occur. When no conflicts occur, the normal merge takes place, ignoring the block. I am exploiting the fact that my data structures will always be pure hashes—there is no need to resort to type checking as is done in the Rails code.

With that, I am much happier and am ready to move onto the next class in my gem. Now that I can generate hashes from javascript files and the directory structure in which they are stored on the filesystem, it is time to get them loaded in the CouchDB store.

The first thing I need for that is a URL:

it "should require a CouchDB URL Root for instantiation" do
lambda { Store.new }.
should raise_error

lambda { Store.new("uri") }.
should_not raise_error
end
I plan to stub out the integration points between my gem and CouchDB so I am not even bothering to try a real URL in the example. If the URL is bad, RestClient will fail for me. To get these examples to pass, I implement this code:
module CouchDesignDocs
class Store
attr_accessor :url
def initialize(url)
@url = url
end
end
end
(the CouchDesignDocs namespace is not needed in the example thanks to an include in the spec_helper.rb as mentioned yesterday)

Next, given a valid Store object and a document hash:
  context "a valid store" do
before(:each) do
@it = Store.new("uri")

@hash = {
'a' => {
'b' => {
'c' => 'function(doc) { return true; }'
}
}
}
end
...
end
I want to create design documents. Again, I am stubbing the interface to CouchDB in the gem. Thankfully, I have much experience with RestClient updates to CouchDB. When uploading the hash in the preconditions, I expect a RestClient.put call like:
    it "should be able to load a hash into design docs" do
RestClient.
should_receive(:put).
with("uri/_design/a",
'{"b":{"c":"function(doc) { return true; }"}}',
:content_type => 'application/json')
@it.load(@hash)
end
And, to make that example pass I write a Store#load method:
    def load(h)
h.each_pair do |document_name, doc|
RestClient.put "#{url}/_design/#{document_name}",
doc.to_json,
:content_type => 'application/json'
end
end
With the Directory and Store classes done, I have reached a good stopping point for the day. I will pick up tomorrow putting it all together.

Thursday, July 9, 2009

Building and Merging Deep Hashes

‹prev | My Chain | next›

Today, I continue work on my couch_design_docs gem. The basic idea remains the same, a javascript file in couch/_design/lucene/transform.js should describe a CouchDB design document:
{
"lucene": {
"transform": <<contents of transform.js>>
}
}
So far, I have a Directory class that converts the directory path, file basename and file contents into an array. Next up, I need a way to convert arrays into deep hashes:
  it "should convert arrays into deep hashes" do
Directory.
a_to_hash(%w{a b c d}).
should == {
'a' => {
'b' => {
'c' => 'd'
}
}
}
end
Running the RSpec example, I get a failure:
cstrom@jaynestown:~/repos/couch_design_docs$ spec ./spec/couch_design_docs_spec.rb
.F.

1)
NoMethodError in 'CouchDesignDocs::Directory should convert arrays into deep hashes'
undefined method `a_to_hash' for CouchDesignDocs::Directory:Class
./spec/couch_design_docs_spec.rb:17:

Finished in 0.006896 seconds

3 examples, 1 failure
I change the message by defining an empty a_to_hash class method:
cstrom@jaynestown:~/repos/couch_design_docs$ spec ./spec/couch_design_docs_spec.rb 
.F.

1)
'CouchDesignDocs::Directory should convert arrays into deep hashes' FAILED
expected: {"a"=>{"b"=>{"c"=>"d"}}},
got: nil (using ==)
./spec/couch_design_docs_spec.rb:25:

Finished in 0.007203 seconds

3 examples, 1 failure
No more changing the message is needed, now I can make it pass. I still rather fancy tail recursion, so I end up with something similar to the other day:
    def self.a_to_hash(a)
key = a.first
if (a.length > 2)
{ key => a_to_hash(a[1,a.length]) }
else
{ key => a.last }
end
end
With that, I am ready to assemble all javascript files into a design docs structure. First up, I make sure that I have more than one javascript file in the fixtures directory:
cstrom@jaynestown:~/repos/couch_design_docs$ find fixtures/
fixtures/
fixtures/a
fixtures/a/b
fixtures/a/b/d.js
fixtures/a/b/c.js
Then I write my example to drive things along:
    it "should assemble all documents into a single docs structure" do
@it.to_hash.
should == {
'a' => {
'b' => {
'c' => 'function(doc) { return true; }',
'd' => 'function(doc) { return true; }'
}
}

}
end
The code that implements this iterates over each .js file in the design docs directory, using the previously built methods to build hashes to be merged together:
    def to_hash
Dir["#{couch_view_dir}/**/*.js"].inject({}) do |memo, filename|
hash = Directory.a_to_hash(expand_file(filename))
memo.merge(hash)
end
end
Simple enough, but wrong:
cstrom@jaynestown:~/repos/couch_design_docs$ spec ./spec/couch_design_docs_spec.rb 
...F

1)
'CouchDesignDocs::Directory a valid directory should assemble all documents into a single docs structure' FAILED
expected: {"a"=>{"b"=>{"c"=>"function(doc) { return true; }", "d"=>"function(doc) { return true; }"}}},
got: {"a"=>{"b"=>{"c"=>"function(doc) { return true; }"}}} (using ==)
./spec/couch_design_docs_spec.rb:48:

Finished in 0.008113 seconds

4 examples, 1 failure
The problem is the merge, which merges at the top level (the "a" key). I want the deepest hashes (the values of "b") to be merged together. Something like this does the trick:
    def deep_hash_merge(h1, h2)
h2.each_key do |k|
if h1.key? k
deep_hash_merge(h1[k], h2[k])
else
h1[k] = h2[k]
end
end
h1
end
It works, but it is not side-effect free. I will likely retry that bit of code tomorrow. My brain is not working so well at this point, so it is a good time to call it a day. Before I quit, leave myself a note:
    it "should assemble all documents into a single docs structure" do
pending "you can do a better job with deep hash merging than that"
@it.to_hash.
should == {
'a' => {
'b' => {
'c' => 'function(doc) { return true; }',
'd' => 'function(doc) { return true; }'
}
}

}
end
Hopefully my tomorrow self will not be too offended.

Wednesday, July 8, 2009

Dem Bones

‹prev | My Chain | next›

I happened to come across Jamie van Dyke's Building a Gem Using BDD article today. It seems the gods of my chain are telling me something and I must listen...

You can probably safely ignore this article, Jamie's write up is far superior...

First up, I install bones:
cstrom@jaynestown:~/repos$ gem install bones
--------------------------
Keep rattlin' dem bones!
--------------------------
Successfully installed bones-2.5.1
1 gem installed
Then I create my gem template:
cstrom@jaynestown:~/repos$ bones create couch_design_docs
Created 'couch_design_docs'
Now you need to fix these files
(in /home/cstrom/repos/couch_design_docs)
README.txt:
* [ 2] [FIXME] (your name)
* [ 3] [FIXME] (url)
* [ 7] [FIXME] (describe your package)
* [ 11] [FIXME] (list of features or problems)
* [ 15] [FIXME] (code sample of usage)
* [ 19] [FIXME] (list of requirements)
* [ 23] [FIXME] (sudo gem install, anything else)
* [ 29] [FIXME] (different license?)

Rakefile:
* [ 22] [FIXME] (who is writing this software)'
* [ 23] [FIXME] (your e-mail)'
* [ 24] [FIXME] (project homepage)'
After fixing as many of those FIXMEs as possible, I move onto implementation. First up is directory parsing. In order to directory parse, the CouchDesignDocs::Directory object is going to need a valid directory. After creating a fixture directory, I begin driving development with spec/couch_design_docs_spec.rb:
require File.join(File.dirname(__FILE__), %w[spec_helper])

describe Directory do
it "should require a root directory for instantiation" do
lambda { Directory.new }.
should raise_error

lambda { Directory.new("foo") }.
should raise_error

lambda { Directory.new("fixtures")}.
should_not raise_error
end
end
Note: I added include CouchDesignDocs to spec/spec_helper.rb so that I could access Directory without the CouchDesignDocs:: namespace.

I implement that code with:
module CouchDesignDocs
class Directory
attr_accessor :couch_view_dir
def initialize(path)
Dir.new(path) # Just checkin'
@couch_view_dir = path
end
end
end
The Dir.new call is made only to raise an exception for invalid paths. It is a quick, cheap way to get the code to behave as desired. Next I create a "valid directory" context and drive the file path expansion needed to build the CouchDB design document JSON structure:
  context "a valid directory" do
before(:each) do
@it = Directory.new("fixtures")
end
it "should list dirs, basename and contents of a file" do
@it.expand_file("fixtures/a/b/c.js").
should == ['a', 'b', 'c', "function(doc) { return true; }\n"]
end
end
After creating the fixtures/a/b/ directory and populating it with a very simple javascript function, I make the example pass with this code:
    def expand_file(filename)
File.dirname(filename).
gsub(/#{couch_view_dir}\/?/, '').
split(/\//) +
[
File.basename(filename, '.js'),
File.new(filename).read
]
end
That will do for a stopping point tonight, I will continue with dem bones gem development tomorrow.