Showing posts with label couchapp. Show all posts
Showing posts with label couchapp. Show all posts

Thursday, April 15, 2010

Uploading Attachments with Node.CouchApp.js

‹prev | My Chain | next›

Last night I was unable to verify that I could upload attachments using a node.couchapp.js. Node.couchapp.js is a node.js reimplementation of couchapp, a framework for creating and maintaining CouchDB applications. When I explored couchapp a while back, I finished off with uploading attachments, which I considered a fairly advanced capability. If node.couchapp.js is capable of uploading attachments, then I will be satisfied that it is capable of writing serious CouchDB applications.

Unfortunately, last night did not go well. The CouchDB logs are no help, so I resort to packet sniffing:
cstrom@whitefall:~/repos/relax$ sudo tcpdump -i lo -n -s 0 -w - port 5984
Trying to upload with the new, node.couchapp.js version, this is what I sniff:
POST /seed/test HTTP/1.1
Host: localhost:5984
Connection: keep-alive
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.371.0 Safari/533.4
Referer: http://localhost:5984/seed/_design/app/_show/update/test
Content-Length: 18379
Cache-Control: max-age=0
Origin: http://localhost:5984
Content-Type: multipart/form-data; boundary=----WebKitFormBoundarywZAoXOCTAcq980gf
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

------WebKitFormBoundarywZAoXOCTAcq980gf
Content-Disposition: form-data; name="_attachments"; filename="e_mummy_salmon_0008.jpg"
Content-Type: image/jpeg

... Image Data ...

------WebKitFormBoundarywZAoXOCTAcq980gf
Content-Disposition: form-data; name="_rev"

10-55c92e5c9b40e823615426e767264bcb
------WebKitFormBoundarywZAoXOCTAcq980gf--
The response from the server is:
HTTP/1.1 409 Conflict
Server: CouchDB/0.10.0 (Erlang OTP/R13B)
Date: Fri, 16 Apr 2010 00:58:02 GMT
Content-Type: text/plain;charset=utf-8
Content-Length: 58
Cache-Control: must-revalidate


{"error":"conflict","reason":"Document update conflict."}
Bah!

With the old, working couchapp version, this is what I see when I upload an image:
POST /eee/test HTTP/1.1
Host: localhost:5984
Connection: keep-alive
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.371.0 Safari/533.4
Referer: http://localhost:5984/eee/_design/relax/_show/upload/test
Content-Length: 18379
Cache-Control: max-age=0
Origin: http://localhost:5984
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary3iardfLQJdZF1Q4L
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

------WebKitFormBoundary3iardfLQJdZF1Q4L
Content-Disposition: form-data; name="_attachments"; filename="e_mummy_salmon_0008.jpg"
Content-Type: image/jpeg

... Image Data ...

------WebKitFormBoundary3iardfLQJdZF1Q4L
Content-Disposition: form-data; name="_rev"

13-a88e99cdd18a1a30d50e3bd59c74b4cb
------WebKitFormBoundary3iardfLQJdZF1Q4L--
And the server replies with:
HTTP/1.1 201 Created
Server: CouchDB/0.10.0 (Erlang OTP/R13B)
Etag: "14-f2afcb2344fa446cb000af4db7c69bea"
Date: Fri, 16 Apr 2010 01:14:19 GMT
Content-Type: text/plain;charset=utf-8
Content-Length: 68
Cache-Control: must-revalidate

{"ok":true,"id":"test","rev":"14-f2afcb2344fa446cb000af4db7c69bea"}
I ediff these two requests and find them identical, aside from the DB, the content boundary and the revision IDs. Identical! What. The. Hell?

Not knowing what else to try, I upload an attachment to the document directly in the CouchDB futon interface. Happily, that also fails. I say happily because I had no idea what else to try at this point.

I still do not know what was wrong with this particular document. To try out upload in node.couchapp.js, I delete the test document with which I had been working. That does not quite work as the document hangs around with an earlier revision number. The second time that I delete it, it finally goes away. In retrospect, I should have investigated this a bit more (perhaps there was a conflict?), but I was grasping at straws. With the document really, really deleted, I retry the upload and:



So I had it working last night, but didn't know it. Not only didn't I know it, but I have not even learned what the problem was. Bummer. Ah well, at least I know that I can do fairly advanced things with node.couchapp.js.

Day #74

Wednesday, April 14, 2010

Uploads with node.couchapp.js Not Quite Working

‹prev | My Chain | next›

Tonight, I would like to see if I can get CouchDB file upload working in node.couchapp.js. I got this working a while back in plain old couchapp. If I end up using node.couchapp.js, I ought to be able to upload images from here as well.

Working with the same design document from the other night, I add an upload form / show document:
var couchapp = require('couchapp');

var ddoc = {_id:'_design/app', shows:{}};
exports.app = ddoc;

ddoc.rewrites = [
{ from: "/foo/:id", to: "/_show/foo/:id", method: "GET" }
];

ddoc.shows.foo = function (doc, req) {
return "<h1>"+ doc['title'] +"</h1><p>Bar.</p>";
};

ddoc.shows.update = function(doc, req) {
var dbname = "seed";

return '' +
'<h1>Upload to <%= title %></h1>' + "\n" +
'' + "\n" +
'<form id="recipe-upload" action="/' + dbname + '/' + doc._id +'" method="post">' + "\n" +
'' + "\n" +
'<p>' + "\n" +
'File to attach:' + "\n" +
'<input type="file" name="_attachments">' + "\n" +
'</p>' + "\n" +
'' + "\n" +
'<p>' + "\n" +
'<button type="submit">Upload</button>' + "\n" +
'<span id="saved" style="display:none;">Saved</span>' + "\n" +
'</p>' + "\n" +
'' + "\n" +
'<input type="hidden" name="_rev" value="'+ doc._rev +'">' + "\n" +
'</form>' + "\n" +
'' + "\n" +
'' + "\n" +
'<script src="/_utils/script/json2.js"></script>' + "\n" +
'<script src="/_utils/script/jquery.js?1.2.6"></script>' + "\n" +
'<script src="/_utils/script/jquery.couch.js?0.8.0"></script>' + "\n" +
'<script src="/_utils/script/jquery.form.js?0.9.0"></script>' + "\n" +
'<script type="text/javascript" charset="utf-8">' + "\n" +
'$("#recipe-upload").submit(function(e) { // invoke callback on submit' + "\n" +
' e.preventDefault();' + "\n" +
' var data = {};' + "\n" +
' $.each($("form :input").serializeArray(), function(i, field) {' + "\n" +
' data[field.name] = field.value;' + "\n" +
' });' + "\n" +
' $("form :file").each(function() {' + "\n" +
' data[this.name] = this.value; // file inputs need special handling' + "\n" +
' });' + "\n" +
'' + "\n" +
' if (!data._attachments || data._attachments.length == 0) {' + "\n" +
' alert("Please select a file to upload.");' + "\n" +
' return;' + "\n" +
' }' + "\n" +
'' + "\n" +
' $(this).ajaxSubmit({' + "\n" +
' url: "/' + dbname + '/' + doc._id +'",' + "\n" +
' success: function(resp) {' + "\n" +
' $("#saved").fadeIn().animate({ opacity: 1.0 },3000).fadeOut();' + "\n" +
' }' + "\n" +
' });' + "\n" +
'});' + "\n" +
'</script>';
};
Ugly as it is, that is pretty much a copy of the upload form that I used successfully earlier. I had to replace the nice templating with Javascript strings. Ick. I can live with it because this is completely throw away / proof-of-concept code.

I sync this document up with my seed DB:
cstrom@whitefall:~/tmp/test_node_couchapp_js$ node ~/repos/node.couchapp.js/lib/bin.js -s -d foo -c http://localhost:5984/seed
Syncing app finished.
And have a look at it in the browser:



So, does it work? Sadly no. Checking the log, I find:
[Thu, 15 Apr 2010 02:11:19 GMT] [debug] [<0.1920.1>] 'POST' /seed/test {1,1}

[Thu, 15 Apr 2010 02:11:19 GMT] [debug] [<0.1920.1>] Minor error in HTTP request: conflict
I have the feeling I am missing come couchapp-specific code in here. I will continue investigating that tomorrow.

For now, I would like to get some better feedback on the upload form. When I press the "Upload" button, the "success" action is fired, animating a "Saved" notification. Clearly it was not saved. Happily, the last go around received a comment from a concerned reader with a pointer for improving my feedback. I incorporate that here:
//...
' success: function(resp) {' + "\n" +
' if(resp.match("ok")){ $("#saved").fadeIn().animate({ opacity: 1.0 },3000).fadeOut();}' + "\n" +
' else if(resp.match("error")){ $("#failed").fadeIn().animate({ opacity: 1.0 },3000).fadeOut();}' + "\n" +
' $("#saved").fadeIn().animate({ opacity: 1.0 },3000).fadeOut();' + "\n" +
' }' + "\n" +
//...
Now I get a "Failed" message appearing next to the save button. I will make use of that tomorrow as I continue to investigate what is needed to get uploads working with node.couchapp.js

Day #73

Monday, April 12, 2010

Quick Intro to node.couchapp.js

‹prev | My Chain | next›

Thanks to some pointers from Mikeal himself, I have some insight on how to work with node.couchapp.js (a node.js reimplementation of couchapp). First up, there seems to be a common starting point:
var couchapp = require('couchapp');

var ddoc = {_id:'_design/app', shows:{}, updates:{}, views:{}, lists:{}};
exports.app = ddoc;
I am not sure if that is enough to create something, but let's give it a try:
cstrom@whitefall:~/tmp/test_node_couchapp_js$ node ~/repos/node.couchapp.js/lib/bin.js \
-s -d foo -c http://localhost:5984/seed
Syncing app finished.
Nice! That's a lot further than I got after an hour of messing about last night. It always helps to have an example or two. Taking a look at the document inside CouchDB's futon, I find:



Interesting, so the simple definition, then export of the ddoc variable is sufficient to create the attributes of the design document.

I wonder if I need to define those attributes at all. Probably not, so I delete them and add a simple show document to make sure that I can do so:
var couchapp = require('couchapp');

var ddoc = {_id:'_design/app', shows:{}};
exports.app = ddoc;

ddoc.shows.foo = function (doc, req) {
return "<h1>Foo!</h1><p>Bar.</p>";
};
I load that design document again:
cstrom@whitefall:~/tmp/test_node_couchapp_js$ node ~/repos/node.couchapp.js/lib/bin.js \
-s -d foo -c http://localhost:5984/seed
Syncing app finished.
(good to know that it can be re-run)

Checking the show document in futon, I find that the extra attributes have indeed been removed without harm:



And, checking that the show document can indeed show documents, I visit the show document proper. I am uploading to the seed database, accessing the _design/app design document, the foo _show document. This translates into a URL of: http://localhost:5984/seed/_design/app/_show/foo. Visiting the URL, I do indeed see the simple show document I hoped to see:



Cool! Well, that was a lot easier than I thought it was going to be after flailing last night. Thanks Mikeal!

Day #71

Friday, February 26, 2010

Lists of Hashes of Hashes in CouchApp

‹prev | My Chain | next›

My exploration of couchapp is nearly at an end. I have a good grasp of edit/updates, of the various javascript callbacks when saving, and even how to upload images. The last thing that I am not quite sure about is deep data structures. More to the point, what about lists of deep data structures?

So far, I have edited only top level attributes of recipes in my cookbook: title, summary, instructions. If I want to use couchapp to edit recipes, however, I must be able to edit the list of tools used or the list of ingredients. Ingredients are especially problematic because we store the ingredient of a recipe as part of ingredient preparation:
{
"brand":"Trader Joe's",
"quantity":1.0,
"order_number":7,
"unit":"jar",
"description":"",
"ingredient":{
"kind":"marinated",
"name":"artichoke hearts"
}
}
If we want the actual ingredient used (marinated artichoke hearts), it is readily available in the "ingredient" attribute. The "ingredient" attribute is just one part of a preparation which includes meta information about the particular ingredient, the amount used and a description of any preparation done to the ingredient (e.g. minced). This might be an overly complex way of storing this information, but we grew into this data structure over the years and are not going to abandon it. The question is, how do we edit it in a web form such that couchapp can readily convert it back-and-forth to JSON for storage in CouchDB?

The most obvious way to accomplish this is to simply expose the JSON to the user in textareas. This is not all that outlandish—we used to edit the same data structure in XML by hand!

Let's presume that we want something a little less painful. Since couchapp will not work without javascript, the ultimate solution to this will likely involve a javascript widget that lists existing ingredient preparations. When an individual item is clicked (or a new item is added) the user would be presented a dialog to edit the ingredient preparation info. When done, the javascript would convert the contents of the dialog into JSON for PUTting into the database.

That seems very do-able, but how about a simple HTML form to edit the entire list?

Couchapp already has the ability to translate deep hash structures into JSON. An HTML form field named preparation-ingredient-name with a value of "artichoke heart" would get translated into the following JSON:
"preparation":{
"ingredient":{
"name": "artichoke heart"
}
}
That is pretty close to what I ultimately want. Perhaps a little tweak will get me what I want...

I create an ingredients.html template that contains fields for two ingredients:
  <tr>
<td><input type="text" name="preparations-0-brand" size="5"></td>
<td><input type="text" name="preparations-0-quantity" size="5"></td>
<td><input type="text" name="preparations-0-unit" size="5"></td>
<td><input type="text" name="preparations-0-ingredient-name" size="5"></td>
<td><input type="text" name="preparations-0-ingredient-kind" size="5"></td>
</tr>
<tr>
<td><input type="text" name="preparations-1-brand" size="5"></td>
<td><input type="text" name="preparations-1-quantity" size="5"></td>
<td><input type="text" name="preparations-1-unit" size="5"></td>
<td><input type="text" name="preparations-1-ingredient-name" size="5"></td>
<td><input type="text" name="preparations-1-ingredient-kind" size="5"></td>
</tr>
I then list each of those fields in couchapp's docForm() that maps form elements to JSON attributes:
<script src="/_utils/script/json2.js"></script>
<script src="/_utils/script/jquery.js?1.2.6"></script>
<script src="/_utils/script/jquery.couch.js?0.8.0"></script>
<script src="<%= asset_path %>/vendor/couchapp/jquery.couchapp.js"></script>
<script type="text/javascript" charset="utf-8">
$.CouchApp(function(app) {

app.docForm("form#update-recipe", {
id : <%= docid %>,
fields: ['title',
"preparations-0-brand",
"preparations-0-quantity",
"preparations-0-unit",
"preparations-0-ingredient-name",
"preparations-0-ingredient-kind",
"preparations-1-brand",
"preparations-1-quantity",
"preparations-1-unit",
"preparations-1-ingredient-name",
"preparations-1-ingredient-kind"
],
beforeSave: function(doc) {
},
success: function(res, doc) {
$('#saved').fadeIn().animate({ opacity: 1.0 },3000).fadeOut();
}
});
});
Amazingly, when I enter some values for a test document, the save succeeds:



It succeeds, but it create a hash of ingredient preparations rather than a list:



It should not be too difficult to convert that into an array. In fact, isn't there a beforeSave() callback that might help? Why yes there is:
//...
beforeSave: function(doc) {
var preparations = [];
for (var prop in doc.preparations) {
preparations.push(doc.preparations[prop]);
}
doc.preparations = preparations;
},
//...
That will collection the entries in the preparations hash, push them onto the preparations local array variable, which then replaces the hash. Now, when I save, the preparations attribute is an array:



That was actually pretty easy!

The best part about this approach, aside from the relative ease of setting up the array, is that couchapp is still able to map the array of preparations back into form fields for editing. This is because access to the name attribute of the first ingredient preparation is the same regardless of hash/array:
recipe['preparations']['0']['ingredient']['name']
If I end up using this solution in a live app I will have to come up with a way of getting the right number of ingredient preparation rows in the table. That is not trivial because the mini-template implementation that couchapp uses does not support loops. I suppose I could pass in the number of ingredients (plus some padding for new ingredients) and use a jQuery on document ready function to clone row zero. I would also need some way of dynamically populating the "fields" attribute in the couchapp docForm() method. I am not too concerned with this so I will probably let it be until/if I do this for real.

Day #26

Thursday, February 25, 2010

How to Upload Files in CouchApp

‹prev | My Chain | next›

Yesterday I failed to get images to upload as CouchDB document attachments using couchapp. This should be do-able given that the Futon administration interface can do it. Just how easy it is remains to be seen...

...which turns out to be fairly easy!

I borrow code from two of CouchDB's javascript libraries:
  • futon.browse.js—defines how the Futon interface submits its file upload form
  • jquery.dialog.js—attaches actions to futon dialogs like the one that uploads files
What I end up with is this in my upload.html template:
<script src="/_utils/script/json2.js"></script>
<script src="/_utils/script/jquery.js?1.2.6"></script>
<script src="/_utils/script/jquery.couch.js?0.8.0"></script>
<script src="/_utils/script/jquery.form.js?0.9.0"></script>
<script src="<%= asset_path %>/vendor/couchapp/jquery.couchapp.js"></script>
<script type="text/javascript" charset="utf-8">
$("#recipe-upload").submit(function(e) { // invoke callback on submit
e.preventDefault();
var data = {};
$.each($("form :input").serializeArray(), function(i, field) {
data[field.name] = field.value;
});
$("form :file").each(function() {
data[this.name] = this.value; // file inputs need special handling
});

if (!data._attachments || data._attachments.length == 0) {
alert("Please select a file to upload.");
return;
}

$(this).ajaxSubmit({
url: "/<%= dbname %>/<%= docid %>",
success: function(resp) {
$('#saved').fadeIn().animate({ opacity: 1.0 },3000).fadeOut();
}
});
});
</script>
Piece by piece, this function disables normal form submission so that ajaxSubmit() can be used instead:
  e.preventDefault();
Next it assembles all of the data in the form into a Javascript object:
  var data = {};
$.each($("form :input").serializeArray(), function(i, field) {
data[field.name] = field.value;
});
$("form :file").each(function() {
data[this.name] = this.value; // file inputs need special handling
});
After checking to see if the data has an empty file upload field, it performs an ajaxSubmit using the current DB and document ID (as supplied by the show function) and defines a simple on-success callback function:
  $(this).ajaxSubmit({
url: "/<%= dbname %>/<%= docid %>",
success: function(resp) {
$('#saved').fadeIn().animate({ opacity: 1.0 },3000).fadeOut();
}
});
To test this out, I create an empty "test" document (more precisely I use this document from yesterday):



Then, I access the upload show function for the test document (http://localhost:5984/eee/_design/relax/_show/upload/test) and upload my favorite avatar image:



The "Saved" animation shows, which leads me to believe that the image has been successfully uploaded. To make sure, I check the document again:



Sure enough, the image is now attached to the CouchDB test document. Yay!

There is still some more that I would do if I wanted this to be ready for every day use—links back to the main edit page, an automatic redirect after upload, etc.—but that is good enough for today. I am just happy to know that it is possible with relatively little effort.

Day #25

Wednesday, February 24, 2010

How Not to Upload Files to CouchApp

‹prev | My Chain | next›

I am continuing my exploration of couchapp today by trying to upload attachments to CouchDB documents. I use attachments in my recipe database to include meal and recipe pictures, so I played with them extensively last year (both command line and with Ruby code).

I am not too sure if it is even possible to upload directly from a web form—I believe that only way that this might work is if CouchDB supports POSTs of multipart/form-data to existing documents. CouchDB's HTTP Document API does not mention this, so I am not hopeful.

Before coding, I explore a bit. I'm pretty sure that CouchDB's futon admin interface supports attachment uploads. Maybe I can re-use (or even copy) that. After creating a test document, I notice that there is an "Upload Attachment..." link in futon. Clicking that link I get this dialog:



The form sure acts like an old fashioned document upload, but inspecting the HTML I find:



The form is not multipart/form-data. It does use a normal file <input> tag and the button is a <button type="submit"> tag (which submits forms just like an <input type="submit"> tag). That is really weird, I have never seen a file upload without a enc="multipart/form-data" attribute. Aside from that, this seems pretty solid—the form is being POSTed to the current document, which should push a new thing onto a sub-collection under that document. The _attachments attribute seems like a perfectly good place for CouchDB to address that.

So maybe this will work...

First I create my shows/upload.js show function:
function(doc, req) {
// !json templates.upload
// !json templates._header
// !json templates._footer
// !code vendor/couchapp/template.js
// !code vendor/couchapp/path.js

return template(templates.upload, {
dbname: 'eee',
docid : doc._id,
rev: doc._rev,
title: doc.title,
asset_path: assetPath(),
header: template(templates._header, {}),
footer: template(templates._footer, {})
});
}
That is very similar to the edit show function that I have used recently, but I take into account that I will need to address the document directly (dbname/docid) in the parameter list.

Next I create the template for this show function:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Recipe Image Upload</title>
<link rel="stylesheet" href="<%= asset_path %>/style/main.css" type="text/css" />
</head>

<body>
<%= header %>

<h1>Upload to <%= title %></h1>

<form id="recipe-upload" action="/<%= dbname %>/<%= docid %>" method="post">

<p>
File to attach:
<input type="file" name="_attachments">
</p>

<p>
<button type="submit">Upload</button>
</p>

<input type="hidden" name="_rev" value="<%= rev %>">
</form>

<%= footer %>
</body>
</html>
No magic in there at all—just like the futon upload I am not specifying multipart/form-data. I am posting directly to the document with only the current document revision number and the uploaded file. Hopefully that will do the trick.

Unfortunately, when I upload I get this:
{"error":"bad_content_type","reason":"Invalid Content-Type header for form upload"}
Dang it.

After some digging, I realize that the "Upload Attachment..." link in futon is not only displaying the form, but also attaching some event listeners. It looks as though some ajaxSubmit() fun is in order. I will give that a whirl tomorrow.

Day #24

Monday, February 22, 2010

CouchApp success() and beforeSave()

‹prev | My Chain | next›

I have my basic recipe edit couchapp work pretty well at this point. There are still a few things that I am not quite sure about:
  • jQuery effects after save (this ought to be easy)
  • defaulting new document IDs to be human readable
  • image upload (I'm not sure this is even possible right now)
  • mapping deep data structures into form variables
  • listing pages to edit
As I say, I doubt that image upload is doable without modifying couchapp—I will leave that to another day (or hope someone else will enlighten me). The others I will work through in order.

First up: jQuery effects upon successful save. This seems fairly trivial. Up until now I have been using tracer bullets to ensure that I am hitting the callbacks I expect in docForm() (a couchapp function that maps form fields to JSON attributes for POST/PUT to the CouchDB database):
// in my edit.html template:
$.CouchApp(function(app) {

app.docForm("form#update-recipe", {
id : <%= docid %>,
fields: ['title', 'summary', 'instructions'],
template: {type: "Recipe"},
beforeSave: function(doc) {
alert("Here!");
},
success: function(res, doc) {
alert("Success!");
}
});
});
Rather than alerting success, I will fade the message in, wait for 3 seconds, then fade out:
$.CouchApp(function(app) {

app.docForm("form#update-recipe", {
id : <%= docid %>,
fields: ['title', 'summary', 'instructions'],
template: {type: "Recipe"},
beforeSave: function(doc) {
// alert("Here!");
},
success: function(res, doc) {
$('#saved').fadeIn().animate({ opacity: 1.0 },3000).fadeOut();
}
});
});
(the #saved id element is a <span> containing the text "Saved")

Now when I click "Save", I see a little "Saved" animation show:



Easy enough (save for the animate() hack that is needed pre-jQuery 1.4)

Onto human-readable IDs...

Right now, I am ending up with document IDs like "09dd3376209faf7aecb08bcbb460d545", which gives me ugly URLs like: http://localhost:5984/eee/_design/relax/_show/edit/09dd3376209faf7aecb08bcbb460d545. Ever since this cooking site was Perl/XML, we had IDs like "2010-02-22-test". I can use another couchapp callback, beforeSave, to get this right.

The beforeSave() callback receives a single argument: the JSON representation of the HTML form. Currently, for a new recipe, this would contain a title, a recipe summary, and instructions. Missing are the date and a pretty ID.

This beforeSave() should add a date (in ISO 8601 format) if one is not present. If the date is not already present, it will set a pretty ID by concatenating the date and a "slugified" title:
$.CouchApp(function(app) {

app.docForm("form#update-recipe", {
id : <%= docid %>,
fields: ['title', 'summary', 'instructions'],
template: {type: "Recipe"},
beforeSave: function(doc) {
var date = new Date();
function zero_pad(num) {
return ((num.toString()).length == 1) ? "0" + num : num.toString();
}
if (!doc.date) {
doc.date = date.getFullYear() + '-' +
zero_pad(date.getMonth() + 1) + '-' +
zero_pad(date.getDate());
doc._id = doc.date + '-' + app.slugifyString(doc.title);
}
}
,
success: function(res, doc) {
$('#saved').fadeIn().animate({ opacity: 1.0 },3000).fadeOut();
}
});
});
(happily, slugifyString is provided by couchapp)

Now, when I save a new document, I see that couchapp has, indeed, used my pretty ID:
[Tue, 23 Feb 2010 03:26:49 GMT] [info] [<0.30439.3>] 127.0.0.1 - - 'PUT' /eee/2010-02-22-test 201
That's a good stopping point. After getting some easy stuff out of the way, I have the feeling that things will get progressively more difficult in the days to come.

Day #22

Sunday, February 21, 2010

CouchApp Create/Update—Now without Errors!

‹prev | My Chain | next›

I ended yesterday with a couchapp error. When I was editing a new document, I got "The document could not be retrieved: missing":



Not unexpectedly, this turns out to be my fault. A peak in the CouchDB log when I accessed the edit page reveals several successful requests (the page itself, CSS, javascript files, etc.):
[Sun, 21 Feb 2010 15:11:01 GMT] [info] [<0.1509.0>] 127.0.0.1 - - 'GET' /eee/_design/relax/_show/edit 304
[Sun, 21 Feb 2010 15:11:01 GMT] [info] [<0.6869.0>] 127.0.0.1 - - 'GET' /eee/_design/relax/style/main.css 304
[Sun, 21 Feb 2010 15:11:01 GMT] [info] [<0.6869.0>] 127.0.0.1 - - 'GET' /_utils/script/json2.js 304
[Sun, 21 Feb 2010 15:11:02 GMT] [info] [<0.6871.0>] 127.0.0.1 - - 'GET' /_utils/script/jquery.js?1.2.6 304
[Sun, 21 Feb 2010 15:11:02 GMT] [info] [<0.6872.0>] 127.0.0.1 - - 'GET' /_utils/script/jquery.couch.js?0.8.0 304

[Sun, 21 Feb 2010 15:11:02 GMT] [info] [<0.6873.0>] 127.0.0.1 - - 'GET' /eee/_design/relax/vendor/couchapp/jquery.couchapp.js 304
And finally, the missing document:
[Sun, 21 Feb 2010 15:11:02 GMT] [info] [<0.6874.0>] 127.0.0.1 - - 'GET' /eee/edit 404
The problem here is in the docForm() function in the edit.html template:
$.CouchApp(function(app) {

var docid = document.location.pathname.split('/').pop();
app.docForm("form#update-recipe", {
id: docid,
fields: ['title', 'summary', 'instructions'],
template: {type: "Recipe"},
beforeSave: function(doc) {
alert("Here!");
},
success: function(res, doc) {
alert("Success!");
}
});
});
I was most likely first exploring edits when I tried calculating the document ID from the URL, which would look something like:
http://localhost:5984/eee/_design/relax/_show/edit/2008-07-12-salmon
Editing a new document would have this URL:
http://localhost:5984/eee/_design/relax/_show/edit
With my current document ID code, couchapp would end up trying to pull back a document with an ID of "edit". So how to do this the idiomatic couchapp way?

To answer that question, I look at the edit show function and template in sofa. The show function contains a docid setting:
  return template(templates.edit, {
docid : toJSON((doc && doc._id) || null),
//...
In the template, sofa uses this value in the docForm() function:
        var postForm = app.docForm("form#new-post", {
id : <%= docid %>,
...
Ah, that explains the use of toJSON() in the show function. For a document ID of 2008-07-21-spinach, toJSON will produce "2008-07-21-spinach", which will be a valid value in the docForm() option hash. If there is no document, the toJSON() function will produce null—another valid javascript value.

While perusing the sofa code, I also noticed that it never passes in existing form values or uses them in templates like I have been:
<label>Title: <input type="text" id="title" name="title" value="" size="50"><%= title ></label>
This is because the docForm() method, which is responsible for mapping form values into JSON to be PUT into the CouchDB database, also looks up existing documents and populates form values for editing.

So my edit.html template become much simpler:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Recipe Edit</title>
<link rel="stylesheet" href="<%= asset_path %>/style/main.css" type="text/css" />
</head>

<body>
<%= header %>

<h1>Editing</h1>

<!-- form to create a post -->
<form id="update-recipe" action="update.html" method="post">

<p>
<label>Title: <input type="text" id="title" name="title" value="" size="50"></label>
</p>

<p>
<label>Summary:<br>
<textarea name="summary" rows="5" cols="80"></textarea>
</label>
</p>

<p>
<label>Instructions:<br>
<textarea name="instructions" rows="15" cols="80"></textarea>
</label>
</p>

<p>
<input type="submit" value="Save &rarr;"/> <span id="saved" style="display:none;">Saved</span>
</p>

</form>

<%= footer %>
</body>
<script src="/_utils/script/json2.js"></script>
<script src="/_utils/script/jquery.js?1.2.6"></script>
<script src="/_utils/script/jquery.couch.js?0.8.0"></script>
<script src="<%= asset_path %>/vendor/couchapp/jquery.couchapp.js"></script>
<script type="text/javascript" charset="utf-8">
$.CouchApp(function(app) {

app.docForm("form#update-recipe", {
id : <%= docid %>,
fields: ['title', 'summary', 'instructions'],
template: {type: "Recipe"},
beforeSave: function(doc) {
alert("Here!");
},
success: function(res, doc) {
alert("Success!");
}
});
});
</script>
</html>
The title, summary, and instructions fields are populated in the web form by couchapp, which uses the fields attribute to drive this. When I view the edit form for the 2008 spinach pie recipe I see all of the fields populated:



In the log, I see the request for the edit template itself, and the subsequent request made for the document to be edited:
[Mon, 22 Feb 2010 03:24:26 GMT] [info] [<0.18563.1>] 127.0.0.1 - - 'GET' /eee/_design/relax/_show/edit/2008-07-21-spinach 200

[Mon, 22 Feb 2010 03:24:27 GMT] [info] [<0.18714.1>] 127.0.0.1 - - 'GET' /eee/2008-07-21-spinach 200
And, when I access the edit show function without a document ID, I get an empty edit form:



Blank and with no error message this time! Removing code and getting better results—I do believe that I am getting the hang of this.

Day #21

Saturday, February 20, 2010

CouchApp Create/Update

‹prev | My Chain | next›

Continuing my exploration of couchapp, I will try to get updates working... well. It took me a bit yesterday, but I was able to submit a normal web form as a PUT of a JSON representation of the form. The JSON was submitted, but rejected by the CouchDB server. Today, I hope to be able to PUT successfully and without losing data.

CouchDB rejected yesterday's PUTs because they did not contain a revision number. CouchDB will reject a PUT to an existing resource with an old or missing revision number as an optimistic locking violation. In other words, if CouchDB cannot verify that the submitted data is a modified version of the current data, the PUT is disallowed.

That should be easy enough to address—adding the revision number to the default values ought to be sufficient. It is going to be a bit of a pain to add all recipe attributes (ingredients, categories, etc.) to the default values so that they are not lost. I'll cross that bridge when I come to it.

Even before I try addressing the revision number issue, I noticed something while perusing source code: the docForm() couchapp method (which maps form fields into JSON to be PUTted) takes an id attribute. Yesterday I was including the ID in the default template attribute:
$.CouchApp(function(app) {

var docid = document.location.pathname.split('/').pop();
app.docForm("form#update-recipe", {
fields: ['title', 'summary', 'instructions'],
template: {type: "Recipe", _id: docid},
beforeSave: function(doc) {
alert("Here!");
},
success: function(res, doc) {
alert("Success!");
}
});
});
I convert that to use the id attribute:
$.CouchApp(function(app) {

var docid = document.location.pathname.split('/').pop();
app.docForm("form#update-recipe", {
id: docid,
fields: ['title', 'summary', 'instructions'],
template: {type: "Recipe"},
beforeSave: function(doc) {
alert("Here!");
},
success: function(res, doc) {
alert("Success!");
}
});
});
I push the update to my DB:
cstrom@whitefall:~/repos/relax$ couchapp push http://localhost:5984/eee
[INFO] Visit your CouchApp here:
http://localhost:5984/eee/_design/relax/index.html
Now when I submit yesterday's form, my tracer bullet still hits (shows the alert("Here!") dialog), but something strange happens. The update is successful:



How on earth did that happen? I still have not added a revision number to the default JSON values. The only thing I changed was the id attribute. So how could that PUT work? Did CouchDB suddenly get all lenient with PUTs?

Checking the update payload, I find the _rev attribute along with other attributes that I did not explicitly set (like prep_time):



The answer is that couchapp does magic with the id attribute. The docForm() method sees id and decides that it is representing an update of an existing resource. As an update, it sets all of the default attributes from the existing document including the _rev. That's pretty freaking cool!

What is even cooler is that couchapp edit templates work for creates just as well as they do for updates. In fact the same form works for both create and update. If the id attribute is not present (e.g. the URL being accessed does not have a document ID at the end), then the form will POST to the database (creating a new record). If the id attribute is set, then the form will PUT to that id in the database.

To verify this, I modify the edit show function slightly to handle null documents:
function(doc, req) {
// !json templates.edit
// !json templates._header
// !json templates._footer
// !code vendor/couchapp/template.js
// !code vendor/couchapp/path.js

return template(templates.edit, {
title: (doc && doc.title),
docid: (doc && doc._id),
asset_path: assetPath(),
summary: (doc && doc.summary),
instructions: (doc && doc.instructions),
header: template(templates._header, {}),
footer: template(templates._footer, {})
});
}
If I access the edit show document without a document ID (http://localhost:5984/eee/_design/relax/_show/edit), I get a blank form:



The first time I submit that form, I see the POST to the DB in the logs:
[Sun, 21 Feb 2010 03:07:54 GMT] [info] [<0.437.0>] 127.0.0.1 - - 'POST' /eee/ 201
If I resubmit, I see the PUT to the document ID what was created by the previous POST:
[Sun, 21 Feb 2010 03:09:01 GMT] [debug] [<0.438.0>] 'PUT' /eee/c7764bf194c11a93d37c91100002787c {1,1}
Ah, I definitely see a use-case for the beforeSave callback in docForm()—creating pretty IDs rather than accepting the default hash supplied by CouchDB.

One thing I still do not know is why I get this document missing alert when I access the edit page without a document ID:



Hopefully, there is an easy way to avoid that. I will pick up there tomorrow.

Aside from that, this couchapp thing rocks!

Day #20

Friday, February 19, 2010

CouchApp Updates (with a Slight Conflict)

‹prev | My Chain | next›

Tonight I continue my exploration of couchapp by attempting to update CouchDB documents. Reading through documentation has left me with the sense that this is rather involved, so I am not sure how far I can get tonight. Only one way to find out!

First up, I copy my recipe show function over to an edit function, removing the textile conversion so that raw textile can be edited:
function(doc, req) {
// !json templates.edit
// !json templates._header
// !json templates._footer
// !code vendor/couchapp/template.js
// !code vendor/couchapp/path.js
// !code lib/super_textile.js

var image;
for (var prop in doc._attachments) {
image = prop;
}

return template(templates.edit, {
title: doc.title,
docid: (doc && doc._id),
asset_path: assetPath(),
summary: doc.summary,
instructions: doc.instructions,
image: image,
header: template(templates._header, {}),
footer: template(templates._footer, {})
});
}
Then some simple form HTML goes into the new templates/edit.html:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Recipe: <%= title %></title>
<link rel="stylesheet" href="<%= asset_path %>/style/main.css" type="text/css" />
</head>

<body>
<%= header %>

<h1>Editing <%= title %></h1>

<!-- form to create a post -->
<form id="udpate-recipe" action="update.html" method="post">

<p>
<label>Title: <input type="text" id="title" name="title" value="<%= title %>" size="50">
</p>

<p>
<label>Summary:<br>
<textarea name="summary" rows="5" cols="80"><%= summary %></textarea>
</label>
</p>

<p>
<label>Instructions:<br>
<textarea name="instructions" rows="15" cols="80"><%= instructions %></textarea>
</label>
</p>

<p>
<input type="submit" value="Save &rarr;"/> <span id="saved" style="display:none;">Saved</span>
</p>

</form>

<img src="../../../../<%= docid %>/<%= image %>" />

<%= footer %>
</body>
</html>
Easy enough, that gives me this edit screen:


Unfortunately, it is not quite that easy. Clicking the Save button does not work:


Forms in couchapp are not old-fashioned POSTs. They need to perform PUTs, DELETEs and POSTs of JSON data. Web browsers won't do that, so some javascript is needed. Fortunately, couchapp takes care of much of the heavy lifting for me. I add this to the bottom of the edit template:
<script src="<%= asset_path %>/vendor/couchapp/jquery.couchapp.js"></script>
<script type="text/javascript" charset="utf-8">
$.CouchApp(function(app) {

var docid = document.location.pathname.split('/').pop();
app.docForm("form#update-recipe", {
fields: ['title', 'summary', 'instructions'],
template: {type: "Recipe", _id: docid},
beforeSave: function(doc) {
alert("Here!");
},
success: function(res, doc) {
alert("Success!");
}
});
});
</script>
I am using a never-gets-old debug through javascript alerts in there—the classics never really go out of style. Actually, that is not so much an example of debugging through alerts as it is an example of using tracer bullets. At least that is what I will tell myself when I try to sleep tonight.

Anyhow...

The two <script> tags pull in some necessary javascript (jquery and some couchapp javascript built with jQuery). With that, I can call the $.CouchApp function to attach RESTful behavior to my form. Specifically, I create a couchapp document form (docForm()) that will convert the form contents into a JSON document to be PUTted into the DB.

The field option supplied to docForm() describes which fields need to be converted from elements into JSON attributes before being PUTted onto the database. The template attribute describes default fields to be submitted (the '_id' is definitely needed to PUT onto the correct document). Lastly, the beforeSave and success callbacks contain my awesome alert() tracer bullets. Once I have the rest of the code hitting them correctly, I can replace the alerts with code to manipulate the JSON document before PUT, and custom code to handle successful updates.

For now, when I submit, I expect to see the beforeSave's "Here!" alert, and a JSON document submitted to the CouchDB database. Unfortunately, what I get is Cannot call method 'db' of undefined at:


It turns out that the following lines are very important:
...</body>
<script src="/_utils/script/json2.js"></script>
<script src="/_utils/script/jquery.js?1.2.6"></script>
<script src="/_utils/script/jquery.couch.js?0.8.0"></script>

<script src="<%= asset_path %>/vendor/couchapp/jquery.couchapp.js"></script>
<script type="text/javascript" charset="utf-8">
$.CouchApp(function(app) {
...
I had ignored those _utils files because I could not find them on the filesystem and just assumed that they were part of sofa. Not so, they are provided by CouchDB itself and, yeah, kinda important.

With those in place, when I submit, I do see the beforeSave() "Here!" tracer bullet, but find that I am not saving the document because of a 409/Conflict:


That actually makes perfect sense—I have not supplied a _rev attribute in the PUT document so CouchDB has no way to apply optimistic locking against the save. That should be easy enough to address, but I will leave that until tomorrow (along with getting the remaining recipe document elements into the form).

Day #19

Thursday, February 18, 2010

Textile and Partial Templates in CouchApp

‹prev | My Chain | next›

Yesterday, I was able to get nearly all of the necessary pieces of a couchapp show page working correctly. Today I will try to do it a little better. Specifically, I would like to figure out rendering Textile via Javascript and re-usable templates (e.g. headers and footers) in CouchDB / couchapp show templates.

I need to be able to display Textile because that is how I edit/store recipe summaries and instructions. On the actual web site, I am simply using Redcloth (by way of Sinatra) to convert for web display.

Happily, I do not have to do much work on Javascript Textile conversion—Jeff Minard and Stuart Langridge have a working demo for doing just this. To add this code to my couchapp, I create a new lib directory and add the "super textile" code to lib/super_textile.js:
/*
* Lifted from http://jrm.cc/extras/live-textile-preview.php
*
* - Jeff Minard (jeff aht creatimation daht net / http://www.jrm.cc/)
* - Stuart Langridge (http://www.kryogenix.org/)
*
*/
function superTextile(s) {
var r = s;
// quick tags first
var qtags = [['\\*', 'strong'],
['\\?\\?', 'cite'],
['\\+', 'ins'], //fixed
['~', 'sub'],
['\\^', 'sup'], // me
['@', 'code']];

// do all sorts of stuff to "r"...

return r;
}
(I make a small change to the code to insert double <br> tags after paragraphs)

To use that function, I pull it into my recipe.js show function with a couchapp !code directive and then, er, use it:
function(doc, req) {
// !json templates.recipe
// !code vendor/couchapp/template.js
// !code vendor/couchapp/path.js
// !code lib/super_textile.js

var image;
for (var prop in doc._attachments) {
image = prop;
}
return template(templates.recipe, {
title: doc.title,
docid: (doc && doc._id),
asset_path: assetPath(),
summary: superTextile(doc.summary),
instructions: superTextile(doc.instructions)
,
image: image
});
}
It is a simple matter to use those new template variables:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Recipe: <%= title %></title>
<link rel="stylesheet" href="<%= asset_path %>/style/main.css" type="text/css" />
</head>

<body>
<h1><%= title %></h1>

<%= summary %>

<%= instructions %>


<img src="../../../../<%= docid %>/<%= image %>" />


</body>
</html>
After uploading the couchapp to my DB, I see nicely formatted textile:


Easy enough.

To add a header and footer, I first create them in the templates directory. I will follow the Rails convention of prefixing partial templates with an underscore, so I create templates/_header.html and templates/_footer.html. To include these in the show function, I again use the couchapp !json directive. To render, I will send() chunks of HTML (first the header, the the main template) to the browser and finally return the footer to be sent last to the browser:
function(doc, req) {
// !json templates.recipe
// !json templates._header
// !json templates._footer

// !code vendor/couchapp/template.js
// !code vendor/couchapp/path.js
// !code lib/super_textile.js

var image;
for (var prop in doc._attachments) {
image = prop;
}

send(template(templates._header, {}));

send(template(templates.recipe, {
title: doc.title,
docid: (doc && doc._id),
asset_path: assetPath(),
summary: superTextile(doc.summary),
instructions: superTextile(doc.instructions),
image: image
}));

return template(templates._footer, {});
}
After pushing my couchapp to my recipes DB, I find this in the browser:


Ugh. Not quite what I was hoping for.

I believe that this is an indication that send() only works in list functions. It makes sense to send data in chunks in a list function—especially if there are many chunks. I expected it to work in the show functions as well. No matter, I can concatenate the template outputs together easily enough (flog scores be damned):
  return template(templates._header, {}) +

template(templates.recipe, {
title: doc.title,
docid: (doc && doc._id),
asset_path: assetPath(),
summary: superTextile(doc.summary),
instructions: superTextile(doc.instructions),
image: image
}) +

template(templates._footer, {});
Now when I load the page, I find:


Much better.

Tomorrow: updating documents with couchapp.

Day #18

Wednesday, February 17, 2010

CouchApp Templates for Showing Documents

‹prev | My Chain | next›

Up tonight is more couchapp fun. Last time around, I was able to install dirt simple pages, including the ability to insert request parameters and document attributes. Tonight I would like to explore the templating feature some with a stretch goal of creating an edit page (updates will come tomorrow).

Templating in couchapp is accomplished via micro-templating from John Resig of jQuery fame. First I need a templates directory (I'm not positive this is needed, but I follow the convention of sofa here):
cstrom@whitefall:~/repos/relax$ mkdir templates
cstrom@whitefall:~/repos/relax$ touch templates/recipe.html
(I am still working in my "relax" couchapp directory from the other day here)

I will populate that template with HTML and other stuff in a bit, but first I create the corresponding show function in the "shows" directory. Specifically, I will create "shows/recipe.js". In that show function, I define the following javascript:
function(doc, req) {
// !json templates.recipe
// !code vendor/couchapp/template.js
return template(templates.recipe, {
title: doc.title
});
}
The !json comment is a couchapp directive, which inserts the templates/recipe.html file into the function at that point and assigns it to a templates.recipe variable/attribute. Similarly, the !code directive inserts code directly from the template.js file into the function. The template.js javascript file contains the micro-templating function template which allows the template() function in the return statement to work.

Now I add HTML to the show function:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Recipe: <%= title %></title>
</head>

<body>
<h1><%= title %></h1>

</body>
</html>
If I have done this correctly (and I my understanding is right), the title of the recipe document (assigned in recipe.js) should be inserted wherever the <%= title %> appears.

I upload the couchapp show function to my recipe database:
cstrom@whitefall:~/repos/relax$ couchapp push http://localhost:5984/eee
[INFO] Visit your CouchApp here:
http://localhost:5984/eee/_design/relax/index.html
I access the show document, applied to a spinach artichoke pie from 2008-07-21 with this URL:http://localhost:5984/eee/_design/relax/_show/recipe/2008-07-21-spinach. The page looks like:



Nice! It worked.

It occurs to me that I have images attached to my recipe documents. To get the image filename, I add the following to the recipe.js code:
function(doc, req) {
// !json templates.recipe
// !code vendor/couchapp/template.js

var image;
for (var prop in doc._attachments) {
image = prop;
}

return template(templates.recipe, {
title: doc.title,
docid: (doc && doc._id),
image: image
});
}
I also added the docid calculation above because it will be needed in the html template:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Recipe: <%= title %></title>
</head>

<body>
<h1><%= title %></h1>

<img src="../../../../<%= docid %>/<%= image %>" />

</body>
</html>
There is probably a better way to get the path (it needs to be relative to the database) than that, but I will figure that out another day. For now, I push the couchapp, reload the web page and:


Nice!

I am not going to reach my stretch goal of starting on the edit template, but before I stop for the day, I do want to make sure I know how to do stylesheets. The generator for couchapp created _attachments/style/main.css. I add the following for the H1 tag:
h1 { border: 2px dotted orange; }
(It should be pretty obvious if that is working!)

Another couchapp supplied javascript file comes in handy here-vendor/couchapp/path.js contains various functions that can generate URL paths (maybe one of them will help my dot infested image tag). The function that I need for stylesheets is assetPath():
function(doc, req) {
// !json templates.recipe
// !code vendor/couchapp/template.js
// !code vendor/couchapp/path.js

var image;
for (var prop in doc._attachments) {
image = prop;
}
return template(templates.recipe, {
title: doc.title,
docid: (doc && doc._id),
asset_path: assetPath(),
image: image
});
}
Using the asset_path local variable in the template looks like:
<link rel="stylesheet" href="<%= asset_path %>/style/main.css" type="text/css" />
After a final re-push of my couchapp design document, I find:



Yup! Ugly orange dots. That is a fine stopping point for tonight. I may do a bit more work on the show before moving onto to the edit. Tomorrow.

Day #17