First Pass at Storing Design Documents to the Filesystem

‹prev | My Chain | next›

After two days of work, I can strike several TODOs from version 1.1 of couch_docs:

• Better command line experience.
  • Should default to current directory.
  • Should print help without args / better format
• Should use the bulk docs
• Should support the !json and !code macros from couchapp
• Should support a flag to only work on design docs (mostly for export).
• Should create the DB if it doesn't already exist

The biggest chunk of work left on the list is to support dumping only CouchDB design documents (or only data documents). That is a hefty piece of work because I currently exclude design documents entirely when dumping to the filesystem:

  # Dump all documents located at <tt>db_uri</tt> into the directory
  # <tt>dir</tt>
  #
  def self.dump(db_uri, dir)
    store = Store.new(db_uri)
    dir = DocumentDirectory.new(dir)
    store.
      map.
      reject { |doc| doc['_id'] =~ /^_design/ }.
      each   { |doc| doc.delete('_rev'); dir.store_document(doc) }
  end

Rather than rejecting design documents, I can tell the design directory object to store them:

  def self.dump(db_uri, dir)
    store = Store.new(db_uri)
    doc_dir    = DocumentDirectory.new(dir)
    design_dir = DesignDirectory.new(dir)
    store.map.each do |doc|
      doc.delete('_rev')
      (doc['_id'] =~ /^_design/ ? design_dir : doc_dir).
        store_document(doc)
    end
  end

With that, an existing spec fails because the design directory instance does not know how to store documents yet:

cstrom@whitefall:~/repos/couch_docs$ spec ./spec/couch_docs_spec.rb 
.....F.....................

1)
NoMethodError in 'CouchDocs dumping CouchDB documents to a directory should ignore design documents'
undefined method `store_document' for #
./spec/couch_docs_spec.rb:78:

Finished in 0.081588 seconds

27 examples, 1 failure

Time to teach it. First up, I get that spec passing by defining an empty DesignDirectory#store_document method.

To get DesignDirectory#store_document to do something substantive, I start to think how I want it to behave. First up, the context—an instance of DesignDirectory and a sample CouchDB design document:

    before(:each) do
      @it = DesignDirectory.new("/tmp")
      @it.stub!(:save_js)
      @design_doc = {
        "id" => "_design/foo",
        "bar" => "function () { 'bar' }",
        "baz" => {
          "json" => "function () { 'baz' }"
        }
      }
    end

I stub out the save_js method because I expect to call that a lot, but I want to be very specific about my expectations. The first such expectation is that the "bar" attribute should be stored in a relative directory "_design/foo/bar" with the value of the "bar" attribute:

     it "should save shallow attributes" do
      @it.
        should_receive(:save_js).
        with("_design/foo/bar",
             "function () { 'bar' }")

      @it.store_document(@design_doc)
    end

I can make that pass with:

    def store_document(doc)
      id = doc['_id']
      doc.each_pair do |key, value|
        self.save_js("#{id}/#{key}", value)
      end
    end

Next I need to actually define the save_js method, but first I make a quick note-to-self that I should not save the _id attribute (that information is contained in the path):

    it "should not store _id"

I find it very effective to make quick notes like this—I loath pending specs and will be sure to address this at the earliest convenience.

For now, I keep focus on the task at hand, defining the save_js method. My context for this requires an instance of DesignDirectory as well as a file object (to write the attribute data) and a stub on FileUtils.mkdir_p (to create the directory structure if it does not already exist). The example describing unrolling an attribute on "_design/foo" to a subdirectory of the same name:

    it "should create map the design document attribute to the filesystem" do
      FileUtils.
        should_receive(:mkdir_p).
        with("/tmp/_design/foo")

      @it.save_js("_design/foo", "bar", "json")
    end

And actually saving the contents to file named "bar.js" (named for the attribute):

    it "should store the attribute to the filesystem" do
      File.
        should_receive(:new).
        with("/tmp/_design/foo/bar.js", "w+")

      @it.save_js("_design/foo", "bar", "json")
    end

I make both of these examples pass with this implementation:

    def save_js(rel_path, key, value)
      path = couch_view_dir + '/' + rel_path
      FileUtils.mkdir_p(path)

      file = File.new("#{path}/#{key}.js", "w+")
      file.write(value.to_json)
      file.close
    end

Being able to save shallow attributes is a decent stopping point for tonight. I will pick up tomorrow with being able to recurse deep data structures (and ignoring things like "_id").

Day #29