Faye as the Persistence Layer in Backbone.js

‹prev | My Chain | next›

Yesterday I was able to override the sync() method in my Backbone.js model to achieve an added layer of persistence. In addition to the normal REST persistence, my model also persists newly created appointments on a Faye pub-sub channel:

          var Appointment = Backbone.Model.extend({
          urlRoot : '/appointments',
          initialize: function(attributes) {
            // ...
            this.faye = new Faye.Client('/faye');
          },
          // ...
          sync: function(method, model, options) {
            if (method == "create") {
              this.faye.publish("/calendars/public", model);
            }
            Backbone.sync.call(this, method, this, options);
          }
        });

As my esteemed Recipes with Backbone co-author pointed out yesterday, it might make sense to switch entirely to Faye for persistence. It is hard for me to wrap my brain around all of the implications for such a change. At the very least, it is going to break my tests, which stub out XHR REST calls (via sinon.js). That aside, will it clean up my backend code?

Only one way to find out and that is to get started. So, in my browser code, I redefine Backbone.sync() to send any sync requests for create, update, delete or read to a faye channel named accordingly:

    var faye = new Faye.Client('/faye');
    Backbone.sync = function(method, model, options) {
      faye.publish("/calendars/" + method, model);
    }

    // Simple logging of Backbone sync messages
    _(['create', 'update', 'delete', 'read']).each(function(method) {
      faye.subscribe('/calendars/' + method, function(message) {
        console.log('[/calendars/' + method + ']');
        console.log(message);
      });
    });

With that, when I reload my funky calendar Backbone application, I see an empty calendar:

There ought to be 10 appointments on that calendar. I just switched persistence transports, so a few other things need to change as well. To figure out where to start, I check Chrome's Javascript console. There, I see that the request for "read" did go out:

That read request comes when the application is initialized—which includes a fetch() of the collection:

      // Initialize the app
      var appointments = new Collections.Appointments;

      new Views.Application({collection: appointments});
      appointments.fetch();

It can be argued that I should not be fetching here, which requires a round trip to the server. The Backbone documentation itself suggests fetching the data in the backend (node.js / express.js in my case). The data can then be interpolated into the page as a Backbone reset() call. Personally, I prefer serving up a static file that shows something almost immediately followed by quick requests to populate the page with useful, actionable information.

To get "actionable" stuff in my currently empty calendar, I need something on the server side to reply to the request on the /calendars/read channel. Doing faye things on the server is relatively easy. I already have the faye node.js adapter hooked in to my express.js application. I can then call to getClient() to gain access to client actions like subscribe():

// Faye server
var bayeux = new faye.NodeAdapter({mount: '/faye', timeout: 45});
bayeux.attach(app);

// Faye clients
var client = bayeux.getClient();

client.subscribe('/calendars/read', function() {
  // do awesome stuff here
});

Now, when the client receives a message on the "read" channel, I can do awesome stuff. In this case, I need to read from my CouchDB backend store:

client.subscribe('/calendars/read', function() {
  //  CouchDB connection options
  var options = {
    host: 'localhost',
    port: 5984,
    path: '/calendar/_all_docs?include_docs=true'
  };

  // Send a GET request to CouchDB
  var req = http.get(options, function(couch_response) {
    console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);

    // Accumulate the response and publish when done
    var data = '';
    couch_response.on('data', function(chunk) { data += chunk; });
    couch_response.on('end', function() {
      var all_docs = JSON.parse(data);
      client.publish('/calendars/reset', all_docs);
    });
  });

  // If anything goes wrong, log it (TODO: publish to the /errors ?)
  req.on('error', function(e) {
    console.log("Got error: " + e.message);
  });
});

This is a bit more work than with the normal REST interface. With pure REST, I could make the request to CouchDB and pipe() the response back to the client. Backbone (more accurately jQuery) itself takes care of parsing the JSON. Here, I have to accumulate the data response from CouchDB and parse it into a JSON object to be published on a Faye channel. I could send back a JSON string, requiring the client to parse, but that feels like bad form. Faye channels can transmit actual data structures, so that is what I ought to do.

Anyhow, I publish to the /calendars/reset channel because that is what the client will do with this information—reset the the currently empty appointments collection:

    window.calendar = new Cal();

    faye.subscribe('/calendars/reset', function(all_docs) {
      console.log('[/calendars/reset]');
      console.log(all_docs);

      calendar.appointments.reset(all_docs);
    });

Upon reloading the page, however, I still see no appointments on the calendar. In the Javascript console, I can see that the /calendar/read message is still going out. I also see that I am getting a response back that includes the ten appointments already scheduled for this month:

So the message is coming back over the /calendars/reset channel as expected. It is the CouchDB query results as expected, but something is going wrong in the call to reset() on the appointments collection. Probably, something related to the "Uncaught ReferenceError: description is not defined" error message at the bottom of the Javascript console.

Digging through the Backbone code a bit (have I mentioned how nice it is to read that?), I find that calls to reset() or add() need to be run through parse() first. Well, not always, just when parse() does something with the data. Something like I had to do with the CouchDB results:

        var Appointments = Backbone.Collection.extend({
          model: Models.Appointment,
          parse: function(response) {
            return _(response.rows).map(function(row) { return row.doc ;});
          }
        });

Anyhow, the fix is easy enough—just run the results through parse():

    faye.subscribe('/calendars/reset', function(message) {
      console.log('[/calendars/reset]');
      console.log(message);

      var all_docs = calendar.appointments.parse(message);
      calendar.appointments.reset(all_docs);
    });

With that, I have my calendar appointments again populating my calendar:

Only now they are being populated via Faye with an assist from overriding Backbone's sync() function.

On the plus side, it was relatively easy to swap out the entire persistence layer in Backbone. A simple (and in this case very small) override of Backbone.sync() did the trick. On the minus side, I had to do a little more work to convert CouchDB responses into real Javascript data structures. That is not a huge negative (and one that I can easily push into a helper function). Still outstanding it how this will affect the entire application. Also, I have the feeling that I could choose faye channel names better. Questions for another day...


Day #143