Tuesday, March 2, 2010

Dumping CouchDB Design Document to the File System

‹prev | My Chain | next›

Today, I continue working through my couch_docs 1.1 TODO:
  • Better command line experience.
    • Should default to current directory.
    • Should print help without args / better format
  • Should use the bulk docs
  • Should support the !json and !code macros from couchapp
  • Should support a flag to only work on design docs (mostly for export).
  • Should create the DB if it doesn't already exist
I am currently working through the last incomplete item—saving design documents.

As of yesterday, I can save shallow design documents:
    it "should store the attribute to the filesystem" do
@file.should_receive(:write).with("json")

@it.save_js("_design/foo", "bar", "json")
end
What happens if the last argument to save_js is a Hash (as is wont in CouchDB design documents)? This example describes that the value should be stored to the file system:
    it "should store hash values to the filesystem" do
File.
should_receive(:new).
with("/tmp/_design/foo/bar/baz.js", "w+")

@it.save_js("_design/foo", "bar", { "baz" => "json" })
end
That example fails with:
1)
Spec::Mocks::MockExpectationError in 'CouchDocs::DesignDirectory saving a JS attribute should store hash values to the filesystem'
<File (class)> received :new with unexpected arguments
expected: ("/tmp/_design/foo/bar/baz.js", "w+")
got: (["/tmp/_design/foo/bar.js", "w+"])
./spec/couch_docs_spec.rb:341:
The save_js method is not traversing down into the Hash when saving. Ah recursion...
    def save_js(rel_path, key, value)
if value.is_a? Hash
value.each_pair do |k, v|
self.save_js(rel_path + '/' + key, k, v)
end

else
path = couch_view_dir + '/' + rel_path
FileUtils.mkdir_p(path)

file = File.new("#{path}/#{key}.js", "w+")
file.write(value)
file.close
end
end
All of my spec are passing, but I have a bit of duplication with the each_pair iterator between save_js() and store_document() (the latter being the entry point for storing documents:
    def store_document(doc)
id = doc['_id']
doc.each_pair do |key, value|
self.save_js(id, key, value)
end

end
There is no need for the iteration in the store_document() entry method, so I convert it to have reasonable default:
    def store_document(doc)
id = doc['_id']
self.save_js("", id, doc)
end
All specs still passing—easy enough.

With that, I have only one last thing to do, implement the spec I left pending to remind myself that I should not be dumping the design document ID (it is already encoded in the directory structure):
Pending:

CouchDocs::DesignDirectory saving a JS attribute should not store _id (Not Yet Implemented)
./spec/couch_docs_spec.rb:298
To remove that spec from the land of the pending, I define it as:
    it "should not store _id" do
File.
should_receive(:new).
with("/tmp/_design/foo/id.js", "w+")

@it.save_js("", "_design/foo", { "id" => "_design/foo"})
end
That fails because the id attribute is being saved:
1)
Spec::Mocks::MockExpectationError in 'CouchDocs::DesignDirectory saving a JS attribute should not store _id'
<File (class)> expected :new with ("/tmp/_design/foo/_id.js", "w+") 0 times, but received it once
./spec/couch_docs_spec.rb:303:
A simple next if k == '_id' statement is all that is required to make that example pass.

With all specs passing and none pending, I am ready to test the new feature on a live database server. First I create the database from a seed directory (making use of the -R couch_docs switch to recreate the database):
cstrom@whitefall:~/repos/couch_docs$ ./bin/couch-docs push http://localhost:5984/test ~/tmp/seed -R  
Then I dump the content of that database back the file system, but in a different location:
cstrom@whitefall:~/repos/couch_docs$ ./bin/couch-docs dump http://localhost:5984/test ~/tmp/dump
If I have done my work correctly, the contents will be identical:
cstrom@whitefall:~/tmp$ find dump/_design/
dump/_design/
dump/_design/meals
dump/_design/meals/views
dump/_design/meals/views/by_year
dump/_design/meals/views/by_year/map.js
dump/_design/meals/views/by_date
dump/_design/meals/views/by_date/map.js
dump/_design/meals/views/count_by_month
dump/_design/meals/views/count_by_month/map.js
dump/_design/meals/views/count_by_month/reduce.js
dump/_design/meals/views/by_date_short
dump/_design/meals/views/by_date_short/map.js
dump/_design/meals/views/by_month
dump/_design/meals/views/by_month/map.js
dump/_design/meals/views/count_by_year
dump/_design/meals/views/count_by_year/map.js
dump/_design/meals/views/count_by_year/reduce.js
dump/_design/recipes
dump/_design/recipes/views
dump/_design/recipes/views/by_date
dump/_design/recipes/views/by_date/map.js
dump/_design/recipes/views/updated_by
dump/_design/recipes/views/updated_by/map.js
....
cstrom@whitefall:~/tmp$ diff -r seed/_design/ dump/_design/
cstrom@whitefall:~/tmp$
Nice!

I still need to hook command line options up to this functionality. That is where I will pick up tomorrow.

Day #30

No comments:

Post a Comment