Rails is a Mature Framework

At a recent B'more on Rails meetup there were no fewer than 7 companies hiring.

Think about that.

In the midst of one of the worst economic climates of the past 100 years, any quality Rails developer has their pick of jobs, likely for excessive amounts of money. What this means to a developer with qualifications is that all of the hard work that you put in 4 years ago learning a then new framework is paying off. You can use the accumulated skills of 4 years to flow -- to push your skills to the maximum as part of exciting teams and be financially rewarded for it. What more could you want from life?

It is a very exciting time for Rails developers.

It shouldn't be. It should be terrifying.

7 companies hiring means that there is significant cash to be made. And if the current crop of developers is not capable of meeting that demand, you can be sure that commodity solutions are imminent.

"Oh, but there is no way to compete with my awesome craftmanship" you say. While that may be true, how to you prove it? More importantly is it really true? Are you providing so much value that your output at $100+/hr is that much better than offshore developers at $40/hr? Do you really think offshore developers incapable of awesome craftsmanship?

One more thing to note: offshore companies aren't stupid. They are already trying to make inroads into Rails development. They are finding that the old strategies that worked so well in the enterprise world are not of much use with most established Rails shops. They will adapt. They will learn agile. They will learn how to participate in agile teams -- maybe not perfectly, but well enough to compete.

So what to do? For now: flow. Make as much money as possible doing interesting things with energetic teams. Enjoy. But...

Start preparing today for the future.

If you love Ruby and Rails and can imagine wanting nothing more from life, now is the time to kick your personal marketing up a notch to keep yourself differentiated from the coming commodity market (might I suggest a nice chain?). If you cannot differentiate yourself, you will find yourself competing (and often losing) against the commodity market for less pay on less interesting work with lesser developers.

Even if you love Ruby and Rails, maybe it is time to learn something new. Not just dabble, but really learn (again, might I suggest a nice chain?).

Personally, I am going all in on node.js. My main motivation? Simpley that it feels like the Rails community of 3 years ago. The community has a bit of a chip on its collective shoulder. The established players scorn it as insufficient for all uses (which is true just as it was for Rails 3 years ago). And there is just a general level of excitement of newness and seemingly unlimited possibilities.

What are you going to be in 4 years? I will be in the flow. But I'll also be getting ready for the next big thing.

Priorities (This Is the End)

‹prev | My Chain | next ???

Effective immediately, I am breaking my chain.

I am incredibly proud of the effort that I have invested in learning over the past year and the year before. This particular chain is 245 days in a row. Every new day is a personal best. Every new day is a personal best that I will likely never surpass.

This chain is very important to me. Very important.

So, by voluntarily breaking it, I am recognizing that my priorities are shifting. For the past two years, my priorities have been:

1. Family
2. Day Job
3. Learning

With the cessation of my chain, I am effectively recognizing that my priorities are now:

1. Family
2. Day Job
3. Cool New Project

Learning obviously, will continue to be very important to me, but, in the Jim Collins tradition:

If you have more than three priorities, then you don't have any

I have an idea. It is a project about which I am passionate.

In short, it is going to be legendary.

Understanding full well the power of public commitment, I furthermore...

Commit to releasing a private beta within two weeks
Monday, October 18

And..

Commit to releasing a public beta within one month
Thursday, November 4

To make that commitment even stronger, I will directly commit to each of the Awesome Coders I Worked With to get this done (sorry in advance for the Twitter spam, guys).

See you on the other side!

Readying Things for the Kids

‹prev | My Chain | next›

Up tonight, I need to start thinking about making things a bit easier for my kids to code my (fab) game.

I expect the entry point that will pull in the kids will be the frames animation that prompted me to write the animate-frames plugin for raphaël.js. The problem right now is that the responsibility for describing the frames is in the game room:

Room.prototype.player_frames = function() {
var standing = [
  { label: "left_leg",
    style: "fill:none;stroke:#000000;stroke-width:0.99999779px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;display:inline",
    d: "M 9.7001312,18.93142 C 9.4644309,22.702657 9.2287306,26.473893 8.9930303,30.24513" },

  // ...
];

var walking = [
  // ...
];

  return [standing, walking];
};

Room.prototype.draw_player = function(player, color) {
  var frames = this.player_frames();

  // set x-y position from player.x, player.y
  return this
    .paper
    .svg_frames(frames)
    .translate(player.x, player.y);
};

It is not completely crazy that the room draws the player. The game board is responsible for holding the reference to the raphaël paper object that draws things. It also needs to keep track of players, makes sure that they do not stray out of the room and sends mouse events to the players.

OK, so that is pretty much a bunch of bull. The room might need to ask the player how the player ought to be drawn, but it has no business holding onto that information.

So first up, I make a player_frames global object, in a separate file, javascript/player_frames.js, that will hold that information. The kids ought to be able to get in there and play with things without being distracted by all the other code. As an intermediate step, I replace the player_frames() call with a reference to the global object:

Room.prototype.draw_player = function(player, color) {
  // set x-y position from player.x, player.y
  return this
    .paper
    .svg_frames(player_frames)
    .translate(player.x, player.y);
};

I then need to pull that global into the page, which is done as a fab.js HTML app:

// ...
      ( SCRIPT, { src:  "/javascript/room.js",
                  type: "application/javascript" } )()

      ( SCRIPT, { src:  "/javascript/player_frames.js",
                  type: "application/javascript" } )()

      ( SCRIPT, { type: "application/javascript" } )
        ( board_js_string )
      ()
// ...

Reloading, I start getting all sorts of undefined errors. It takes me a bit, but I eventually realize that one of the problems with HTML (fab) apps is that the server needs to be restarted when changes are made—even to HTML pages.

Once I figure that out, I get things working. Ultimately, I make the Player responsible for the player frames:

Room.prototype.draw_player = function(player, color) {
  // set x-y position from player.x, player.y
  return this
    .paper
    .svg_frames(player.frames())
    .translate(player.x, player.y);
};

Ah, much better.

That will suffice for tonight. Up tomorrow, I need to get some of the changes to the animate-frames plugin back into the main repository. I would also like to make sure that everything still works with the latest version of fab.js.

Never a shortage of things to learn.


Day #245

Preventing Unnecessary Loss of Limbs in Games

‹prev | My Chain | next›

The players in my (fab) game are suffering some serious injuries:



Loss of limbs is not occurring immediately or even after a long time roaming about the game room. No, the player's legs are becoming detached after bouncing into each other repeatedly:



Crazy. Where does one even start to track down a problem like that?

Well, since I have made changes to it recently and since it is controlling the player walking animation, I start with animate-frames, my animation plugin for raphaël.js.

In there, I add an offsets() method to the individual frames making up the animation. The offsets calculate the x-y differences between the various strokes of the pen that comprise the frame (e.g. a pen stroke for a foot, leg, body, etc). The x-y differences are between each of the elements and the last element.

  Frame.prototype.offsets = function() {
    var diffs = []
       ,main  = this.getBBox();
    
    for (var i=0; i<this.elements.length-1; i++) {
      var box = this.elements[i].getBBox();
      diffs.push([main.x - box.x, main.y - box.y]);
    }

    return diffs;
  };

The individual offsets should always remain the same—a player's foot should stay 16 pixels away from the body if it was first drawn 16 pixels away. Looking at the first offset when the player first enters the room, I find:

[
0: -6.787272029958575
1: -16.76693459682566
]

After moving the player about the room for a bit, I find:

[
0: -6.787412453957089
1: -16.766913780265497
]

Almost exactly the same. I am not sure about the differences (in the 10,000ths decimal place). Is that caused by simple rounding errors? Maybe timing differences between stop() calls of individual elements is sufficient to cause that problem? Is that small a change even a problem?

Before investigating further, I try colliding my player with another player in the room. Then, I check the offsets again:

[
0: -6.039369453957079
1: -16.56494217026568
]

That is a much more dramatic change. Banging players together some more furthers the offset change:

[
0: -5.144486633957058
1: -16.511516330265806
]

Whatever the problem, it seems that the bouncing of players is the best place to continue my investigation.

But wait...

If I know the expected offset of the individual elements in a frame, can I use them to sync the frame back up to where it is supposed to be?

I think that I can run through each offset, calculate the difference between the current and original offset, then translate the affected frame element by the tiny offset it needs to get back to the right place:

  Frame.prototype.tighten = function() {
    var offsets = this.offsets();
    for (var i=0; i<offsets.length; i++) {
      var x_diff = offsets[i][0] - this.original_offsets[i][0]
         ,y_diff = offsets[i][1] - this.original_offsets[i][1];

      this.elements[i].translate(x_diff, y_diff);
    }
  };

Yay! That works just as expected. A player falling to pieces gets put back together (tightened up) with a single call.

To put that to practical use, I call it at the end of the translate() method of each Frame object. And, just like that, my players keep all of their limbs.

That is not an entirely satisfactory solution—I never did solve why limbs were flying away in the first place. Rounding errors seems a reasonable explanation, but reasonable explanations do not elicit learning the way that digging deep does. Something to investigate another day.


Day #244

Gah! I Still Don't Know Closures

‹prev | My Chain | next›

I ended a big refactoring of animate-frames last night with some less than DRY code:

  Frame.prototype.remove = function () {
    for (var i=0; i<this.elements.length; i++) {
      this.elements[i].remove();
    };
  };

  Frame.prototype.show = function() {
    for (var i=0; i<this.elements.length; i++) {
      this.elements[i].show();
    };
  };

  Frame.prototype.hide = function() {
    for (var i=0; i<this.elements.length; i++) {
      this.elements[i].hide();
    };
  };

  Frame.prototype.stop = function() {
    for (var i=0; i<this.elements.length; i++) {
      this.elements[i].stop();
    };
  };

Let's see if I can DRY that up a bit.

For each of those methods (remove(), show(), hide(), and stop()), I set the prototype property of the Frame object to that repetitive function:

  var methods = ['remove', 'show', 'hide', 'stop'];
  for (var i=0; i<methods.length; i++) {
    var method = methods[i];
    Frame.prototype[methods[i]] = function () {
      console.log(method);
      for (var j=0; j<this.elements.length; j++) {
        this.elements[j][method]();
      };
    };
  }

The only funky thing in there is accessing the method property on each element. element.stop is the same thing as element['stop']—both of them are functions that stop raphaël.js elements. Putting parentheses on either invokes the function: element.stop() and element['stop']().

At least I think that will work. Sadly, when I load that up in my (fab) game, the players do not move at all. Checking the output in the Javascript console, I see that the method being called is always stop()—even when the show() and hide() methods are being invoked.

What gives? Am I messing things up with the property method call?

Actually, no. I have messed up, but what I have messed up here is my Javascript closures. The anonymous function that is being assigned to the Frame.prototype method is itself a closure:

  var methods = ['remove', 'show', 'hide', 'stop'];
  for (var i=0; i<methods.length; i++) {
    var method = methods[i];
    Frame.prototype[methods[i]] = function () {
      console.log(method);
      for (var j=0; j<this.elements.length; j++) {
        this.elements[j][method]();
      };
    };
  }

But the context that it is enclosing includes that var method. My mistake is that the context, the value of var method changes with each iteration through the list of methods. At the end of the iteration, the method variable is set to 'stop'.

What this means is that every one of the methods that I just defined is a stop() method. I do define Frame.prototype.remove(), Frame.prototype.show(), etc. But, when each is called, they use the method from the enclosed context—which is forever more set to "stop".

So what can I do to get the context to stick? I need to add more context—another function:

  var methods = ['remove', 'show', 'hide', 'stop'];
  for (var i=0; i<methods.length; i++) {
    Frame.prototype[methods[i]] = function(method) {
      return function () {
        console.log(method);
        for (var j=0; j<this.elements.length; j++) {
          this.elements[j][method]();
        };
      };
    }(methods[i]);
  }

For each method that I am defining, I set the prototype method to the return value of calling the anonymous function with the method name:

    Frame.prototype[methods[i]] = function(method) {
      // ...
    }(methods[i]);

Now, when I return the same function as earlier, the method variable is in a new, separate context—one that will be remembered when the methods are called. To state is more plainly—in order to ensure that a value in a Javascript closure is what you think it is, it must be defined inside another, enclosing function.

That was unexpectedly difficult. Still, I learned me some good closure. I think I will stop there and move on to resolving the weird bounce issue in animate-frames tomorrow.


Day #243

The Great (Lucky) Refactor

‹prev | My Chain | next›

I got a new feature added to animate-frames last night. Getting there made me realize that the code is a bit messy so tonight I refactor!

Animate-frames is a plugin for raphaël.js that iterates through a series of raphaël/SVG frames as the frames move about the paper. I am using this in my (fab) game to draw and animate players as they move about the game room. I am going to refactor in the game before extracting the changes back out into the plugin.

The refactoring is necessary because many of the methods in the plugin look like:

     stop: function() {
       var frames = this.list;
       for (var i=0; i<frames.length; i++) {
         for (var j=0; j<frames[i].length; j++) {
           frames[i][j].stop();
         }
       }
     }

The class is responsible for managing the frames and the parts that make up the frames:



(the parts that make up the frames are the players' body, legs, and feet — each is a different SVG path)

I could make that much cleaner if the frames methods were just responsible for knowing about the frames, but made a separate class responsible for dealing with the pieces. In that case, this:

     stop: function() {
       var frames = this.list;
       for (var i=0; i<frames.length; i++) {
         for (var j=0; j<frames[i].length; j++) {
           frames[i][j].stop();
         }
       }
     }

...would become this:

    stop: function() {
      for (var i=0; i<this.list.length; i++) {
        this.list[i].stop();
      }
    }

Ah, much better. No more worrying about what that j variable was iterating over. No more trying to discern which of those 6 lines are important. Just one clear iteration through each frame in the animation, telling each frame to stop().

The corresponding stop method on my Frame object is then:

  Frame.prototype.stop = function() {
    for (var i=0; i<this.elements.length; i++) {
      this.elements[i].stop();
    };
  };

Again, nice and compact. Just one iteration making it very clear what is going—working through each part that makes up the frame and telling it to stop.

I end up working through half a dozen similar methods. All sorts of weird naming becomes easier (translate_object() becomes just translate() on the Frame object). Initialization code gets separated in much easier to read ways. It is great.

And then something amazing happens....

I load it up in the browser and it just works! I have no tests (note to self: future idea for a chain post) that helped me support the refactoring. Git diff reports:

cstrom@whitefall:~/repos/my_fab_game$ gd --stat
 javascript/raphael-animate_frames.js |  192 +++++++++++++++++++---------------
 1 files changed, 108 insertions(+), 84 deletions(-)

That is a lot of non-trivial change. I had to intentionally break the code to make sure I was running the updated code (I was).

Amazing? Am I that good?

Absolutely not. That was pure luck and I damn well know it. The coding gods should have punished me severely for even attempting it. I give them thanks for going easy on me tonight, but I know I have a doozy of a refactoring fail in my future.

One thing I do notice is that I repeat myself in the simple raphael methods:

  Frame.prototype.remove = function () {
    for (var i=0; i<this.elements.length; i++) {
      this.elements[i].remove();
    };
  };

  Frame.prototype.show = function() {
    for (var i=0; i<this.elements.length; i++) {
      this.elements[i].show();
    };
  };

  Frame.prototype.hide = function() {
    for (var i=0; i<this.elements.length; i++) {
      this.elements[i].hide();
    };
  };

  Frame.prototype.stop = function() {
    for (var i=0; i<this.elements.length; i++) {
      this.elements[i].stop();
    };
  };

I also notice that my players become disjointed after colliding too often:



I will pick back up tomorrow with DRYing up my code after the great refactor and trying to solve the detached limb problem.


Day #142

Workaround and Hacks

‹prev | My Chain | next›

Tonight, I continue working through a new feature in animate-frames plugin for raphaël.js. This plugin iterates through a series of raphaël/SVG frames as the frames move about the paper. I am using this in my (fab) game to draw and animate players as they move about the game room:



I found last night that it is not possible to animate position changes of elements drawn from the path() method (which my players are). I would like to animate collisions between elements.

My plan tonight is to hide all of the frames, then draw a replacement circle that bounces. At the end of the bounce, I will remove the bounced circle and re-show one of the frames. To accomplish this, I add a third parameter to the animate() method in animate-frames—the same parameter that is in core raphaël describing the easing:

    animate: function(new_attrs, ms, easing) {
    }

If that argument is not present, I will fallback to the old behavior (strict translation of coordinates). If that argument is present, then I do as I describe above:

    animate: function(new_attrs, ms, easing) {
      if (new_attrs.cx || new_attrs.cy) {
        var x = new_attrs.cx || this.getCenter().x
           ,y = new_attrs.cy || this.getCenter().y;
        if (easing) {
          this.hide_all_frames();
          // replace the frames with a circle at the same location
          var c = this.paper.circle(this.getCenter().x,
                                    this.getCenter().y,
                                    20);
          var self = this;
          c.animate({cx: x, cy: y}, ms, easing, function () {
            // when the animation of the circle is done, remove
            // the circle, move the frames to the end point and show
            // the first frame again
            c.remove();
            self.translate(x, y);
            self.show_frame(self.list[0]);
          });
        }
        else {
          this.translate(x, y, ms);
        }
      }
    }

That almost works.

The circle appears, but the frames do not disappear. They stick around and keep moving.

After a bit of a search, I track the problem down to the stop() method in the Player class. Specifically, when a player stops (e.g. when it is about to bounce), the raphael object is not being told to stop:

Player.prototype.stop = function () {
  //TODO -- re-enable
  // this.avatar.stop();

  this.x = this.avatar.getBBox().x;
  this.y = this.avatar.getBBox().y;
};

Hunh. That's weird. I wonder why it is disabled....

After re-enabling, I find out why it was disabled—there is no stop() method in animate-frames. So I add one:

    stop: function() {
      var frames = this.list;
      for (var i=0; i<frames.length; i++) {
        for (var j=0; j<frames[i].length; j++) {
          frames[i][j].stop()
      }
    }

I really need to break down and create a Frame class. I iterate over frames and frame elements far too often.

I will leave refactoring for another night. For now, I would be happy just to get things working.

Happily, that was the last change needed:



I could definitely make that animation a little nicer, but, for a proof of concept it will suffice. Tomorrow, I will prettyfy things a bit—both the animation and the code.


Day #241

Can't Animate Raphaël path Positions

‹prev | My Chain | next›

I made some good progress last night making animate-frames more raphaël.js-like. Before I push those changes back into the main repository, I would like to explore easing in the animate() method.

Ultimately, the animate() method in animate-frames is calling the translate_object() method, which animates movement with animateAlong():

    translate_object: function(frame, x, y, ms, easing) {
      // offset between end coordinates and current location
      var x_diff = x - frame[frame.length-1].getBBox().x;
      var y_diff = y - frame[frame.length-1].getBBox().y;

      for (var i=0; i<frame.length; i++) {
        var obj = frame[i];
        // calculate path starting from absolute coordinates and moving
        // relatively from there by the offset
        var p = "M " + obj.getBBox().x + " " + obj.getBBox().y +
               " l " + x_diff          + " " + y_diff;

         // animate along that path
        if (ms && ms > 50)
          obj.animateAlong(p, ms);
        else
          obj.translate(x_diff, y_diff);
      };
    }

My first instinct is to add and easing parameter to animateAlong(). Sadly, according to the documentation (and the code), the method signature only takes path, ms, rotate, and callback (invoked after the animation completes).

Dang.

So maybe I can use animate() instead of animateAlong():

         // animate along that path
        if (ms && ms > 50)
          obj.animate({x: x + x_diff, y: y + y_diff}, ms, easing);
        // ...

But when I do that, my players don't move at all. Grr.. what gives?

To find out, I break the problem down to basics. Using a raphaël paper object, r, I create a circle and animate its center x-coordinate:

c = r.circle(200, 200, 10).attr({stroke: "#333", "stroke-width": 4, "fill": '#333'})
c.animate({cx: 0}, 5000)

That works. The circle moves from 200, 200 to 0, 200 over the course of 5 seconds.

Next I try drawing an object with SVG path information—the same way that the objects inside my animations are being drawn:

o = r.path("m 27.970621,11.503807 c 0.09878,4.794487 -3.625825,9.254299 -8.244694,9.888541 C 15.282994,22.185422 10.583894,19.322284 9.1090954,14.943452 7.4780985,10.605964 9.2934491,5.2861677 13.201573,2.948285 c 3.892493,-2.53638571 9.452051,-1.6375587 12.397375,2.008774 1.523084,1.7954543 2.381825,4.1636216 2.371673,6.546748 z").attr({fill: '#090'})
o.animate({cx: 50}, 5000)

That does not work. The player stays at its original coordinates.

I can animate certain other attributes, such as the fill color:

o.animate({fill: '#00f'}, 5000)

From this, I conclude that it is not possible to animate location elements on objects drawn via paths. That is not too surprising based on my earlier investigation. Not surprising, but still disappointing.

I am not quite ready to abandon easing. I really want to use it to animate collisions (the "bounce" easing is particularly nice). Since I cannot animate-with-easing the SVG paths that make up my frames, I think that I ought to be able to hide the frames, show another object that can be animated. That is not perfect, but it ought to suffice for my purposes.

I will get started on that tomorrow.


Day #240

Better Raphaël Plugins

‹prev | My Chain | next›

Up tonight, I try to get a few more raphaël.js methods into raphael-animate-frames. This plugin for raphael.js cycles through a series of frames, giving the appearance of animating. Additionally, the frames can move together and do other things that normal raphael.js objects can. There are many raphaël things that it cannot do. I am going to start tonight by trying to add an animate() method.

The documentation for animate explains that not all attributes can be animated. I will push that to the extreme and say that only 2 elements can be animated: cx and cy (center x and center y).

Some attributes just do not make sense to be animated. Do the attributes apply to all elements in an animation frame or only some? Actually, reading through the list, I ought to be able to support many eventually, but for now, I just need cx and cy:

Player.prototype.bounce_to = function(x, y) {
  this.stop();

  var self = this;
  this.avatar.animate({cx: x, cy: y}, 500, "bounce");
  setTimeout(function(){ self.mid_bounce = false; }, 1000);

  this.x = x;
  this.y = y;
};

I can get a quick and dirty implementation of that by making animate() a simple wrapper around the translate() method:

    animate: function(new_attrs, ms) {
      if (new_attrs.cx || new_attrs.cy) {
        this.translate(new_attrs.cx || this.getCenter().x,
                       new_attrs.cy || this.getCenter().y,
                       ms);
      }
    }

I will not get easement by using translate(), but it will suffice for tonight.

Up next, I need animate-frames to support a node attribute so that I can get decent collision detection. That turns out to be relatively easy:

    add: function(frame_list) {
      for (var i=0; i<frame_list.length; i++) {
        this.list.push(this.draw_object(frame_list[i]));
      }
      this.node = this.list[0][this.list[0].length-1].node;
    }

The this.list[0] references the first frame of elements. Using this.list[0].length-1 gives me the last elment drawn in the frame. This seems a reasonable thing—the last thing drawn in a frame is very likely going to be the most prominent feature which makes it ideal to serve at the node attribute.

With that, I can set a boolean on the node indicating that it is the "player" node. If the player node collides with another player node, then collision detection kicks in:

Player.prototype.attach_avatar = function(avatar) {
  var self = this;
  this.avatar = avatar;

  // ...

  avatar.node.is_player_circle = true;

  // ...

  avatar.onAnimation(function(){
    // find the colliding element (c_el)

    if (!self.initial_walk &&
        !self.mid_bounce &&
        c_el.is_player_circle &&
        c_el != self.avatar.node) {
      self._bounce_away(c_x, c_y);
    }
  });
};

That is a fine stopping point for tonight. I will pick back up with easing tomorrow.


Day #239

My Dogfood Stinks

‹prev | My Chain | next›


Having resurrected the animate-frames fork of my (fab) game last night, I am ready to play again with my raphaël.js plugin: animate-frames. If I recall correctly, it is very much a work in progress, but is in good enough shape for a demo.

Unfortunately, the my (fab) game code has evolved significantly since last I played with this code. For one thing, the animate-frames plugin does not have a single node on which various attributes can be set:

//... 

  // Mark node as player element (for collision detection)
  avatar.node.is_player_circle = true;

  // ...

For now, I will simply comment out references to node.is_player_circle. I can go back later to get collision detection working well.

Unfortunately, I run into a another problem—the remove() method, used when a player is dropped, is not defined in animate-frames:

Player.prototype.quit = function() {
  this.avatar.remove();
  this.label.remove();
  delete this.x;
  delete this.y;
  delete this.id;
};

This, I can address relatively easily by adding a method that removes all frames being animated as well as each element in each frame:

    remove: function() {
      for (var i=0; i<frames.length; i++) {
        for (var j=0; j<frames[i].length; j++) {
          frame[j].remove();
        };
      }
    }

This is browser code, so I cannot rely on forEach(). Actually, this is a indication that I ought to be using coffeescript.

At any rate, it works.

Until I hit another problem—collision detection does not work at all. Players simply pass through each other. Eventually, I track this down to the initial_walk attribute on a player. Collision detection does not kick in until the first walk in the room is complete.

The only place that I see initial_walk even mentioned as being set to false is a comment:

Player.prototype.walk_to = function(x, y) {
  // ...

  // TODO self.initial_walk = false after walk has started

  // ...
};

It has been a while since I last fiddled with the animiate-frames stuff, so I am not even going to bother digging through the history of how I got in this state. Instead, I have a peak back in master since I know that collision detection is working in there. Sure enough, it is set:

Player.prototype.walk_to = function(x, y) {
  // ...

  var self = this;
  this.avatar.animateAlong(p, time, function(){
    self.initial_walk = false;
  });

  // ...
};

Grr... that (animateAlong()) is another method that I do not have in animate-frames. I can simulate what I need with setTimeout()—2 seconds after the player first starts walking in the room, wait 2 seconds then mark the initial walk as done:

  var self = this;
  setTimeout(function() {self.initial_walk=false;}, 2*1000);

Sadly, that is not the last of my woes. Last up is the lack of support for the animate() method when the players bounce away from a collision:

Player.prototype.bounce_to = function(x, y) {
  this.stop();

  var self = this;
  this.avatar.animate({cx: x, cy: y}, 500, "bounce");
  setTimeout(function(){ self.mid_bounce = false; }, 1000);

  this.x = x;
  this.y = y;
};

Blech. I workaround this tonight by using the translate() method, but ick. This has been a tough night eating my own dog food. In order, I wanted the node attribute, the animateAlong() method and the animate() method. I am not sure how realistic the node attribute is, but I really ought to be able get the animateAlong() and animateAlong() methods in there.

Tomorrow.


Day #238

A Fork of a Fork of a Fork

‹prev | My Chain | next›

I have a problem. A git problem.

A ways back, I got my animate frames plugin for raphaël.js working in my (fab) game. It was beautiful. Unfortunately, it was all very organic.

I forked the animate-frames stuff off a vows.js fork. It got me to the point that I could extract the animate frames out, but I never got it to the point that I could merge it back into master. I ultimately abandoned the fab.js work that drove the vows.js stuff that eventually spawned the animate frames work. But now I want just the animate frames stuff.

So how do I get that without the old fabjs and vowsjs stuff tagging along? It is only a half dozen commits, so I could cherry pick them. Since this seems like something that I will have to do again where cherry picking is not feasible (e.g. where there are many more than 6 commits involved), I think I will spend at least a little time trying to find another way.

So I head to google and find suggestions to use git merge (probably wouldn't work in my case) and, more interestingly, git checkout (it never would have occurred to me to try that).

But ultimately, I come across the advice to rebase --onto. Using that, I can rebase a branch, animate-frames in this case, to master, starting from the vowsjs fork. The command for that is blessedly simple:

cstrom@whitefall:~/repos/my_fab_game$ git rebase --onto master vowsjs animate-frames
First, rewinding head to replay your work on top of it...
Applying: Pull in the latest raphael-animate-frames
Applying: Switch room to raphael-animate-frames for drawing player avatar.
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging javascript/room.js
CONFLICT (content): Merge conflict in javascript/room.js
Failed to merge in the changes.
Patch failed at 0002 Switch room to raphael-animate-frames for drawing player avatar.

When you have resolved this problem run "git rebase --continue".
If you would prefer to skip this patch, instead run "git rebase --skip".
To restore the original branch and stop rebasing run "git rebase --abort".

The command may be simple, but a lot has gone on in master since the animate-frames work. Happily that turns out to be the only problem. I address the issue in the javascript/room.js, put it back into the rebase staging area with git add javascript/room.js, then send rebase back on it merry way:

strom@whitefall:~/repos/my_fab_game$ git add javascript/room.js
cstrom@whitefall:~/repos/my_fab_game$ git rebase --continue
Applying: Switch room to raphael-animate-frames for drawing player avatar.
Applying: Use the raphael-animate-frames.js lib
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging html/board.html
Applying: Replace circle avatar calls with svg-frames equivalent
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging javascript/player.js
Applying: Use latest raphael-animate-frames
Applying: Use center() in raphael animate frames
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging javascript/player.js

With that, I have my animate-frames branch believing that it was forked from master only a few days ago:



The whole point of this is so that I can play with animated players on top of my node-dirty work from last night. I remain fairly certain that I am ready to move on with node-dirty instead of CouchDB, but a bit of play will help get the remaining doubt down to an acceptable minimum.

So I create a dirty-svg branch (naming is even more fun with dirty than with CouchDB!) and merge the animate frames branch into to it:

cstrom@whitefall:~/repos/my_fab_game$ git merge animate-frames 
Merge made by recursive.
 html/board.html                      |    1 +
 javascript/player.js                 |   30 +++-----
 javascript/raphael-animate_frames.js |  136 ++++++++++++++++++++++++++++++++++
 javascript/room.js                   |   58 ++++++++++++++-
 4 files changed, 202 insertions(+), 23 deletions(-)
 create mode 100644 javascript/raphael-animate_frames.js

Since both are (now) very recent forks from master, the merge goes through without objection. And, just like that, I am ready to start playing with animated-frame players on top of a node-dirty backend.

Tomorrow.


Day #237

Down and node-dirty

‹prev | My Chain | next›

Having exhausted (for the moment) my exploration of the upcoming fab.js version 0.5, I feel ready to move onto the next big thing in my (fab) game. First, I merge my topic branch back into master and push everything to the github repository. With that, I am ready to get dirty:

cstrom@whitefall:~/repos/my_fab_game$ git co -b dirty
Switched to a new branch 'dirty'

Yup. In this branch, I am going to replace my beloved CouchDB with Felix Geisendörfer's node-dirty as the player store.

First up, I install it via npm:

npm install dirty
npm info it worked if it ends with ok
npm info version 0.1.27-12
npm info fetch http://registry.npmjs.org/dirty/-/dirty@0.9.0.tgz
npm info fetch http://registry.npmjs.org/gently/-/gently-0.8.0.tgz
npm info install dirty@0.9.0
npm info install gently@0.8.0
npm info activate dirty@0.9.0
npm info activate gently@0.8.0
npm info build Success: dirty@0.9.0
npm info build Success: gently@0.8.0
npm ok

It ends with "ok" so it worked :)

Next, I work through the player store object replacing node-couchdb calls with node-dirty calls. The player get() method used to be a wrapper around the node-couchdb getDoc() method:

   get: function(id, callback) {
     Logger.debug("[players.get] trying to get: " + id);
     db.getDoc(id, function(err, res) {
       if (err) {
         Logger.warn(JSON.stringify(err));
       }
       callback(res);
     });
   }

Now it can send the results of a node-dirty get() directly to the supplied callback:

   get: function(id, callback) {
     Logger.debug("[players.get] trying to get: " + id);
     callback(db.get(id));
   }

Really, there is no need for the get() method to be async (callback-based) anymore. It could just as easily block for the node-dirty get(). I leave it as-is to minimize API changes—at least for now.

In addition to getting players, I need to update the saving of them as well. The node-couchdb saveDoc():

        db.saveDoc(player);

...becomes a node-dirty set():

        db.set(status.id, player);

In the node-couchdb version, I could rely on the id attribute in the document to store the proper ID in CouchDB. In node-dirty, I need to be explicit.

Lastly, I need to be able to drop players. In the node-couchdb version, I had to get the player before it could be removed (to satisfy optimistic locking):

  drop_player: function(id) {
     Logger.info("players.drop_player " + id);
     this.faye.publish("/players/drop", id);
 
     this.get(id, function(player) {
       Logger.debug("[players.drop_player] " + inspect(player));
       if (player) db.removeDoc(id, player._rev);
     });
   }

There is no need for that in node-dirty. To be precise, there is no support for locking in node-dirty, which could certainly prove an issue at some point. For now, I will put that concern aside and remove players with a simple rm() call:

  drop_player: function(id) {
     Logger.info("players.drop_player " + id);
     this.faye.publish("/players/drop", id);
     db.rm(id);
   }

With that, I am ready to fire up my game. Sadly, I find:

Caught uncaughtException: Error: Cannot find module 'constants'
    at loadModule (node.js:275:15)
    at require (node.js:411:14)
    at Object.<anonymous> (/home/cstrom/.node_libraries/.npm/dirty/0.9.0/package/lib/dirty/dirty.js:6:17)
    at Module._compile (node.js:462:23)
    at Module._loadScriptSync (node.js:469:10)
    at Module.loadSync (node.js:338:12)
    at loadModule (node.js:283:14)
    at require (node.js:411:14)
    at Object.<anonymous> (/home/cstrom/.node_libraries/.npm/dirty/0.9.0/package/lib/dirty/index.js:1:80)
    at Module._compile (node.js:462:23)

My uncaught exception handler catches another one:

// Don't crash on errors
process.on('uncaughtException', function (err) {
  console.log('Caught uncaughtException: ' + err.stack);
});

It prevents the crash, but the question remains: how do I prevent the exception in the first place. If node cannot find the 'constants' module, perhaps I can install it from npm?

cstrom@whitefall:~/repos/my_fab_game$ npm install constants
npm info it worked if it ends with ok
npm info version 0.1.27-12
npm ERR! ! 404 ! 
npm ERR! ! 404 ! It seems like 'constants' is not in the registry
npm ERR! ! 404 ! You should bug the author to publish it.
npm ERR! ! 404 ! 
npm ERR! Error: 404 Not Found: constants
...
npm not ok

Guess not.

I recently discovered that yes, you can read the node.js source code, so that is what I do. And I find that ENOENT can be accessed from process.binding('net'):

node> process.binding('net').ENOENT
2

After replacing the constants.ENOENT references with process.binding('net').ENOENT, everything works! So I do the right thing and submit a pull request for the real node-dirty.

That one minor (and easily resolved) issue aside, the move to node-dirty was pure win. I no longer require a separate CouchDB server to run my (fab) game, and it seems as though it will be more than sufficient to meet my needs—the game will crap out long before 1 million players start to impact node-dirty.

I am still a bit concerned about record locking, but I will leave that question for another day.

Day #236

Conditional (fab) HTML Apps

‹prev | My Chain | next›

Up tonight, I would like to get rid of the idle timeout alert() in my (fab) game. I have already done this once when I was playing with express. Now I would like to do the same on version 0.5 of fab.js.

First up, I update the good-bye function in the client to redirect (instead of alert()-ing) to the login page with an appropriate message:

    var goodbye = function() { 
      window.location = window.location.protocol + '//' + 
        window.location.host + 
        window.location.pathname + 
        '?message=You have been logged out.'  ; 
    }; 
    player_list = new PlayerList(me, room, {onComplete: goodbye});

In the express.js version of the code, I had used flash/session to accomplish this. There is no flash in fab (yet), so query strings will have to suffice.

With that, I need to grab the message out of the query string and insert it into the HTML. Since I am using (fab) HTML apps to render the HTML, I need to make the message display a (fab) app as well:

  (route, /^\/board/)
    ( HTML )
      ( head )
      ( BODY )
        ( info )
        ( FORM, { id: "login", method: "get" } )
        // form stuff
        ()
        ( DIV, { id: "room-container" } )()
      () // BODY

    () // HTML
  ()

Fortunately, I am getting pretty good at this:

function info(write) {
  return write(function(write, head) {
    Logger.info(inspect(head));
    return write;
  });
}

Well, maybe not too good. I know how to write a (fab) app that gets called with a response (write) stream. I also know that returning that response stream with a callback will allow any other upstream apps (e.g. the FORM and other HTML apps) to write to the response stream as well. Lastly, I know that the callback in this situation is called with the response stream, the request header (head) and the request body (which I do not need).

What I do not know (or at least do not remember off the top of my head) is where in the head the URL and query string are buried. To find out I access the /board resource with a query string of foo=bar and see what Logger.info tells me:

[INFO] { method: 'GET'
, headers:
   { host: 'localhost:4011'
   , connection: 'keep-alive'
   , accept: 'application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'
   , 'user-agent': 'Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.8 Safari/534.7'
   , 'accept-encoding': 'gzip,deflate,sdch'
   , 'accept-language': 'en-US,en;q=0.8'
   , 'accept-charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
   }
, url:
   { href: '//localhost:4011/board?foo=bar'
   , slashes: true
   , host: 'localhost:4011'
   , port: '4011'
   , hostname: 'localhost'
   , search: '?foo=bar'
   , query: 'foo=bar'
   , pathname: ''
   , capture: []
   }
}  

Ah, in the url.query property. I know how to extract query parameters. node.js provides a querystring parse function to accomplish this:

function info(write) {
  return write(function(write, head) {
    var q = require('querystring').parse(head.url.query);
    if (q.message)
      write('<div id="info">' + q.message + '</div>');
    return write;
  });
}

And it turns out, that is all I need. When the idle timeout is broadcast, the goodbye function is called in the client, which redirects to the login page with a message and I see:



Easy enough! I think I am finally getting the hang of the new version of fab.js.

Day #235

Javascript Separation of Concerns with Functions

‹prev | My Chain | next›

Up tonight, a bit of code clean-up. The main game.js file in my (fab) game has grown to 350+ lines of code as it now maintains a CouchDB, establishes a faye server (and some server side listeners), and responds to various resources necessary to drive the game. It has been difficult working with the code, so I start moving some of it to the lib directory.

I can write my dirt-simple Logger in lib/logger.js as:

var puts = require( "sys" ).puts;

Logger = {
  level: 1,

  debug:  function(msg) { if (this.level<1) puts("[DEBUG] " + msg); },
  info:   function(msg) { if (this.level<2)  puts("[INFO] " + msg); },
  warn:   function(msg) { if (this.level<3)  puts("[WARN] " + msg); },
  errror: function(msg) { if (this.level<4)  puts("[ERROR] " + msg); }
};

module.exports = Logger;

The module.exports statement at the end is a commonjs mechanism for describing what will be imported into the main game.js code when it requires the module:

var Logger = require("./lib/logger");

Logger.info("Starting up...");

Easy enough.

I factor out the player store code as well:

var couchdb = require('node-couchdb/lib/couchdb'),
    client  = couchdb.createClient(5984, 'localhost'),
    db      = client.db('my-fab-game');

// Player local store
var players = (require("./lib/players")).init(db);

The code is starting to look better and the extraction process is going smoothly until I hit the

  // attach the extension ensuring player messages come from the same
  // client that originally added player to the room
faye_server.addExtension(require("./lib/faye_server_auth"));

The problem with this faye extension is that it accesses the player store, which is now isolated in its own module.

Blech. It was almost nicer when everything was just in one big namespace and everything had access to everything. But, deep down, I know that will lead to poor separation of concerns. So I stop my pining for the bad old days and seek a mechanism to allow the message authorization to lookup players.

I could require the players module inside the message authorization module. I opt against that because the player code does much more (maybe too much more) than just lookup players in CouchDB—it also establishes faye subscriptions to update the data store in response to various published messages.

Instead, I pass a function (this is javascript, after all) into the message authorization init() method that can be used to look up players:

  // attach the extension ensuring player messages come from the same
  // client that originally added player to the room
  var auth = require("./lib/faye_server_auth").init(function () {
    players.get.apply(players, arguments);
  });
  faye_server.addExtension(auth);

I have to be careful to return the object itself in init() because I need that object when calling faye_server.addExtension():

  init: function(player_get) {
    this.player_get = player_get;
    return this;
  }

I can then use the player_get callback inside the message authorization module thusly:

    this.player_get(message.data.id, function(player) {
      // Do stuff with the player
    });

With that, I have my game.js source down to 180 lines—100 of which is HTML (fab) apps. I am not completely convinced that passing a function in by init() is inherently better than requiring the data store in the message authorization module. Still, if the message authorization module had direct access to the player store, the main code body would still have to pass in the database name. Unsure, I will leave it as-is, but make a note to reconsider in the future.

That is a nice stopping point for the night. Up tomorrow, I think that I am ready to merge back into master and return to some raphaël.js fun.


Day #234

More DRY with fab.js v0.5

‹prev | My Chain | next›

Up tonight I try to put my new fab.accept—an app written for the upcoming v0.5 release of fab.js—to some good use. Currently in my (fab) game, I have a player /status route that looks like:

 ( route, /^\/status/ )
    ( accept.HTML )
      ( PRE )
        ( player_status )
      ()
    ()
    ( accept.PLAIN )
      (undefined, {headers: { "Content-Type": "text/plain"}})
      ( player_status )
    ()
  ()

Depending on the requested content type, the status of the players in the game are served up as HTML or plain text. The problem is that the HTML being served up is very small. Ideally I would like to serve it up as part of the same HTML template that the main player board is using:

  (route, /^\/board/)
    ( HTML )
     ( HEAD )
      ( TITLE )( "My (fab) Game" )()
      ( LINK, { href:  "/stylesheets/board.css",
                media: "screen",
                rel:   "stylesheet",
                type:  "text/css" } )
      // Lots more style and script
      ( BODY )
      // The actual body of the request

But how to re-use that? Back when I was first messing about with DRYing up my HTML, I found that I could have two different routes inside the same template (a rather unique fab.js feature):

  ( HTML )
    ( HEAD )
      ( TITLE )( "My (fab) Game" )()
      ( LINK, { href:  "/stylesheets/board.css",
                media: "screen",
                rel:   "stylesheet",
                type:  "text/css" } )
      // Lots more style and script
    ()
    ( BODY )

      (route, /^\/board/)
        // the game board HTML
          
      ()

      (route, /^\/status/)
        ( PRE )
          ( player_status )
        ()
      ()

I can no longer make use of that because the plain text response cannot include the HTML + HEAD + BODY tags that wrap both routes. Instead, I need to re-use the HTML, but only when the /status resource is requesting HTML. It turns out the upcoming version version of fab.js makes this easy by squirreling away partials in a local variable:

with ( fab )
with ( html ) head =

  ( fab )
    ( HEAD )
      ( TITLE )( "My (fab) Game" )()
      ( LINK, { href:  "/stylesheets/board.css",
                media: "screen",
                rel:   "stylesheet",
                type:  "text/css" } )
      // Lots more style and script
    ()
  ();

I can then use that partial in both the /board route:

  (route, /^\/board/)
    ( HTML )
      ( head )
      ( BODY )
        ( FORM, { id: "login", method: "get" } )
          ( LABEL )
            ( "Name" )
            ( INPUT, { type: "text", name: "player" } )
          ()
          ( INPUT, { type: "submit", value: "Play" } )
        ()
        ( DIV, { id: "room-container" } )()
      () // BODY

    () // HTML
  ()

...and in the player /status route (body only when the client accepts HTML):

  ( route, /^\/status/ )
    ( accept.HTML )
      ( HTML )
        ( head )
        ( BODY )
          ( PRE )
            ( player_status )
          ()
        ()
      ()
    ()
    ( accept.PLAIN )
      (undefined, {headers: { "Content-Type": "text/plain"}})
      ( player_status )
    ()
    ( "not found", { status: 404 } )
  ()

That is a surprisingly effective way to get simple partial support. Once again, I am left thinking that there may just be something to this upcoming version of fabjs.


Day #233

An npm Package of My Very Own (fab.accept)

‹prev | My Chain | next›

I have yet to create a npm package of my own. The fab.accept (fab) app from the other day affords me the perfect opportunity. It is an extremely small app, but that is one of the ideas behind the upcoming version of fab.js—small, modular apps.

First, I create a github repository for fab.accept. With that, I am ready to get started building my node package.

During this process, I make extensive use of the npm help subsystem. For instance, I read through npm help json to figure out how to write my package.json file:

{ "name": "fab.accept"
, "version": "0.0.1"
, "author": "Chris Strom"
, "description": "Simple (fab) app to respond to Accept HTTP headers"
, "homepage": "http://github.com/eee-c/fab.accept"
, "engines": ["node >= 0.2.0"]
, "main": "lib/main" }

There is also a dependencies attribute supported by npm. I opt not to use that here because it would require a version of fab.js (0.5) that is not yet in the npm registry.

After creating a lib/main.js with the code from the other day, I am ready to install locally:

cstrom@whitefall:~/repos/fab.accept$ npm install .
npm info it worked if it ends with ok
npm info version 0.1.27-12
npm info install fab.accept@0.0.1
npm info activate fab.accept@0.0.1
npm info build Success: fab.accept@0.0.1
npm ok

Nice! It took me a little while to root through the npm help json documentation to figure out what I needed, but once I had that sorted out, it was pretty easy to get my package ready to be installed.

After that, I use it in my (fab) game to make sure that it is working. I had left some debug code in there and accessed an array incorrectly. Once I have those issues resolved, I am ready to publish to the npm registry.

I am not sure what sort of authorization I need (do I need to send an email to get an account?). Only one way to find out:

cstrom@whitefall:~/repos/fab.accept$ npm publish .
npm info it worked if it ends with ok
npm info version 0.1.27-12
npm ERR! Failed PUT response undefined
npm ERR! Error: Cannot insert data into the registry without authorization
npm ERR! See: npm-adduser(1)
npm ERR!     at request (/home/cstrom/.node_libraries/.npm/npm/0.1.27-12/package/lib/utils/registry/request.js:38:15)
npm ERR!     at PUT (/home/cstrom/.node_libraries/.npm/npm/0.1.27-12/package/lib/utils/registry/request.js:129:34)
npm ERR!     at Object.publish (/home/cstrom/.node_libraries/.npm/npm/0.1.27-12/package/lib/utils/registry/publish.js:49:7)
npm ERR!     at /home/cstrom/.node_libraries/.npm/npm/0.1.27-12/package/lib/publish.js:15:14
npm ERR!     at /home/cstrom/.node_libraries/.npm/npm/0.1.27-12/package/lib/cache.js:178:11
npm ERR!     at cb (/home/cstrom/.node_libraries/.npm/npm/0.1.27-12/package/lib/utils/graceful-fs.js:28:9)
npm ERR!     at node.js:769:9
npm ERR! try running: 'npm help publish'
npm ERR! Report this *entire* log at <http://github.com/isaacs/npm/issues>
npm ERR! or email it to <npm-@googlegroups.com>
npm not ok

Beautiful. It might not have worked, but it tells me exactly what I need to do:

cstrom@whitefall:~/repos/fab.accept$ npm adduser
Username: eee-c
Password: 
Email: npm@eeecooks.com
npm info it worked if it ends with ok
npm info version 0.1.27-12
npm info adduser Authorized user eee-c
npm ok

And now...

cstrom@whitefall:~/repos/fab.accept$ npm publish .
npm info it worked if it ends with ok
npm info version 0.1.27-12
npm WARN Sending authorization over insecure channel.
npm WARN Sending authorization over insecure channel.
npm WARN Sending authorization over insecure channel.
npm info publish done with upload
npm info publish uploaded
npm ok

Damn, that was pretty easy. This npm stuff is all right!

The only thing left for me is to install from the npm registry to make sure it all works. Before I can do that, I need to remove my locally installed package:

cstrom@whitefall:~/.node_libraries$ npm rm fab.accept
npm info it worked if it ends with ok
npm info version 0.1.27-12
npm info uninstall safe to uninstall: fab.accept-0.0.1
npm info uninstall fab.accept-0.0.1 complete
npm ok

I also need to remove the previously installed package from the cache:

cstrom@whitefall:~/.node_libraries$ npm cache clean
npm info it worked if it ends with ok
npm info version 0.1.27-12
npm ok

If I do not remove from the cache, then I am not testing the downloaded package. I have no reason to believe that anything will not work, but, as long as I am testing this, I might as well test what I think I am testing.

With that, I can install from the npm registry:

cstrom@whitefall:~/.node_libraries$ npm install fab.accept
npm info it worked if it ends with ok
npm info version 0.1.27-12
npm info fetch http://registry.npmjs.org/fab.accept/-/fab.accept-0.0.1.tgz
npm info install fab.accept@0.0.1
npm info activate fab.accept@0.0.1
npm info build Success: fab.accept@0.0.1
npm ok

Success!

This npm is pretty darn cool. I went from no package to published package in a single night. I cannot ask for much more than that.


Day #232

Picking Up After a Long Time Away

‹prev | My Chain | next›

Since I have been at Ruby DCamp this weekend, I thought it would make some sense to actually do some Ruby code. Way back when, I started this chain with the idea that I would come up with a mechanism to update EEE Cooks. The data resides in a CouchDB database. At least one way that I might accomplish that goal is via my couch_docs gem.

So I pick back up with my couch_docs. It has been a while, but I remember that I was partly through adding attachment support. How to pick it back quickly?

Happily, I left myself some clues:



Actually, I left myself a complete plan of what I need to accomplish:



Pending specs are incredibly powerful for this kind of thing. I use them even when I am not abandoning gems. If I am in the middle of working a problem at the end of the day, I will brain dump the remaining work in the form of pending specs.

The first pending spec that I will work today is:

      it "should guess the mime type"

That is nice and compact—no block or anything other cermony. Just a descrition of what I need to implement.

Unfortunately, when I look at the other specs in here, there is a ton of setup involved:

      it "should connect attachments by sub-directory name (foo.json => foo/)" do
        everything = []
        @it.each_document do |name, contents|
          everything << [name, contents]
        end

        everything.
          should include(['baz_with_attachments',
                         {'baz' => '3',
                            "_attachments" => { "spacer.gif" => {"data" => @spacer_b64} } }])
      end

That feels like a lot of work (especially considering there is additional setup in before(:each) blocks). Instead of rewriting immediately, I try to work with the same stuff for my should-infer-mime-time pending spec:

      it "should guess the mime type" do
        JSON.stub!(:parse).
          and_return({ "baz" => "3",
                       "_attachments" => {
                         "spacer.gif" => "asdf",
                       }
                     })

        everything = []
        @it.each_document do |name, contents|
          everything << [name, contents]
        end

        everything.
          should include(['baz_with_attachments',
            { 'baz' => '3',
              "_attachments" => {
                "spacer.gif" => { "data" => @spacer_b64},
                "baz.jpg" => "asdf",
                "content_type" => "image/gif"
              }
            }])
      end

Holy wow! That is a crazy amount of code just to test that content type. It passes, but clearly it is time to refactor.

With all tests passing, I can focus on the main culprit of my testing complexity, the each_document method:

    def each_document
      Dir["#{couch_doc_dir}/*.json"].each do |filename|
        id = File.basename(filename, '.json')
        json = JSON.parse(File.new(filename).read)

        if File.directory? "#{couch_doc_dir}/#{id}"
          json["_attachments"] ||= { }
          Dir["#{couch_doc_dir}/#{id}/*"].each do |attachment|
            next unless File.file? attachment

            attachment_name = File.basename(attachment)
            type = mime_type(File.extname(attachment))
            data = File.read(attachment)
            json["_attachments"][attachment_name] =
              {
                "data" => Base64.encode64(data).gsub(/\n/, '')
              }
            if type
              json["_attachments"][attachment_name].
                merge!({"content_type" => type})
            end
          end
        end

        yield [ id, json ]
      end
    end

Yikes! I choose to believe that I knew there was way too much complexity the last time I worked this and that I intended to refactor. That is just silly.

Not coincidentally, the behavior for which I wrote my test is part of the complexity in each_document. Specifically, converting the file on the filesystem (including knowing the file type) can be factored out into a smaller method:

    def file_as_attachment(file)
      type = mime_type(File.extname(file))
      data = File.read(file)

      attachment =  {
        "data" => Base64.encode64(data).gsub(/\n/, '')
      }
      if type
        attachment.merge!({"content_type" => type})
      end

      attachment
    end

That is still a largish method, but there is no looping and only a single conditional. That is pretty good. And much easier to test for the content type:

      it "should guess the mime type" do
        File.stub!(:read).and_return("asdf")
        Base64.stub!(:encode64).and_return("asdf")

        @it.file_as_attachment("spacer.gif").
          should == {
            "data"         => "asdf",
            "content_type" => "image/gif"
          }
      end

That also makes for quicker BDDing of other mime types. Once I have everything working to my satisfaction, I load my fixtures into a test DB via the command line:

cstrom@whitefall:~/repos/couch_docs$ ./bin/couch-docs push http://localhost:5984/couch_docs_test ./fixtures/
Updating documents on CouchDB Server...

...and, checking in the DB:



Yay! The algorithm being used for mime inference is extraordinarily rudimentary, but it is well isolated in the mime_type method for future improvement. My main takeaway from today was that I was able to pick up the code very quickly, even after a long time away, thanks to pending specs. Those same specs are also pretty good for refactoring.


Day #231

Spike fab.accept

‹prev | My Chain | next›

Up today, I would like to explore code re-use of (fab) apps in the upcoming version 0.5 of fab.js. In my (fab) game, I have a dashboard resource that returns that list of players currently in the room:

      (route, /^\/player_status/)
        ( PRE )
          ( player_status )
        () // PRE
      () // route

Looking at this is the browser, I see:



Simple enough.

Now that I think about it though, I really prefer to see this from curl. It is nice to have it in HTML so I would just as soon not have to choose from one or the other. Aha! Code reuse opportunity!

I create a new /status resource that produces a simple text response:

  ( route, /^\/status/ )
    (undefined, {headers: { "Content-Type": "text/plain"}})
    ( player_status )
  ()

I can access that via curl and see:

cstrom@whitefall:~/repos/my_fab_game$ curl http://localhost:4011/status
bob
fred

Cool! Quick, dirty resuable (fab) apps are quick and easy to roll. But...

There is the problem of using two different routes to represent the same thing. I would much prefer requesting a single resource and inspectint Accept headers so that something like his would work:

  ( route, /^\/status/ )
    ( accept, "text/html" )
      ( PRE )
        ( player_status )
      ()
    ()
    (accept, "text/plain")
      (undefined, {headers: { "Content-Type": "text/plain"}})
      ( player_status )
    ()
  ()

For this, I will need to write a new (fab) app—fab.accept.

I start as simple as possible:

fab.accept = function() {
  function accept( write, mime_type ) {
    return fab.stream( function( yes ) {
      return fab.stream( function( no ) {
        return write( function( write, head ) {
          // console.log(inspect(head));
          return ( head.headers.accept == mime_type ? yes : no )( write, head );
        })();
      });
    });
  }

  return accept;
}();

That is close to boiler-plate (fab) code for ternary apps. Like all (fab) apps, it returns a function (accept()) that also returns a function. To get the conditional branch, that function also returns a function. If the condition matches (e.g. the Accept header matches), then the first application,
denoted by the yes local variable, is executed. Otherwise, the no application is executed (e.g. the next fab.accept or a 404).

I did have to use a console.log() (commented out), but only because I can never remember the attributes of the headers in fab.js / node.js. With that, I am ready to test fab.accept from the command line. There is no easy way to get curl to request plain text or HTML. Instead, I need to set the "Accept" HTTP header attribute with the -H option:

cstrom@whitefall:~/repos/my_fab_game$ curl -H "Accept: text/html" http://localhost:4011/status
<pre>bob
fred

</pre>

Nice, the HTML version of the resource is served up in response to "Accept: text/html". Can I get the plaint text version?

cstrom@whitefall:~/repos/my_fab_game$ curl -H "Accept: text/plain" http://localhost:4011/status
bob
foo

Nice!

I still have to polish off fab.accept (and it would be nice to have some tests), but that is a nice stopping point for the night. It is pretty cool that it so easy to write these little things!


Day #230

How to Write Multiple Strings to a Single fab.stream

‹prev | My Chain | next›

I have been struggling a bit with an edge case of fab.stream—a (fab) app from the upcoming version 0.5 of fab.js. To explore that difficulty, I break down the problem into a very small (fab) app:

with ( fab )
with ( html )

( fab )

  // Listen on the FAB port and establish the faye server
  ( listen, 0xFAB )

   ( PRE )
     ( callback_test )
   () // /pre

();

As I found out last night, HTML (fab) apps that wrap content (like <pre>) do so by writing to the response stream, then reading from the upstream app response (e.g. callback_test above), then writing the closing tag to the response stream. The end result being that callback_test has upstream data in the response when it gets invoked.

It makes perfect sense but ended up causing trouble in certain situations. Specifically, when the upstream app writes multiple times to the response stream, then the HTML app will send multiple close tags.

This does not occur for simple apps. For instance if I grab a string foo from a function and write it to the response stream, then write the hard-coded string "bar" to the response stream like so:

function non_stream_test(write) {
  var foo = function_that_returns_foo();
  return write(foo)
    ("bar");
}

...It just works. Accessing this resource via curl returns:

cstrom@whitefall:~$ curl -N http://localhost:4011/
<pre>foobar</pre>

The foo + bar test is wrapped in the <pre> tag as expected.

Of course, I could also just as easily written to the response stream once like so:

function non_stream_test(write) {
  var foo = function_that_returns_foo();
  return write(foo + "bar");
}

My problem is that I think the first way is just so cool.

Anyhow.

As I mentioned, the above cases just work. Problems arise when dealing will callbacks—for example, looking up a list of players in a database. Since I am using Javascript, the database lookup expects to be called with function that it can invoke once it has retrieved the list of player. This is Javascript, so it is quite common.

In that case, I cannot return a write() of data. I only have access to the data inside the callback. Calling return inside the callback will return from the callback, not the function that originally requested the data. Fortunately, in v0.5 of fabjs, there is fab.stream to handle this.

Consider a silly callback that invokes the supplied function with a single argument, "foo":

function callback_with_foo(fn) { fn("foo") }

I can then get that value, along with "bar" by doing something like:

function callback_test(write) {
  return write(function(write) {
    return fab.stream(function(stream) {
      callback_with_foo(function (foo) {
        stream(write(foo));
        stream(write("bar"));
        stream(write());
      });
    });
  });
}

This is the situation in which I will get a second closing <pre> tag:

cstrom@whitefall:~$ curl -N http://localhost:4011/
<pre>foo</pre>bar</pre>

Each stream(write()) still has the closing tag in the upstream just waiting to be sent back to the browser. And merrily sent it is.

For the past 3 nights, I have been banging my head against the proverbial wall trying to figure out a way around this. Today, it finally occurred to me that there is no need. I cannot think of a use case in which I would want to write twice in a callback and, even if I could, then it might warrant a bit more firepower than (fab) HTML apps. A perfectly cromulent solution is to simply accumulate the string data and then write it once:

function callback_test(write) {
  return write(function(write) {
    return fab.stream(function(stream) {
      callback_with_foo(function (foo) {
        stream(write(foo + "bar"));
        stream(write());
      });
    });
  });
}

(the second stream() with an empty write() signals to stream that it is done sending data)

With that, I get my properly wrapped text from a callback:

cstrom@whitefall:~$ curl -N http://localhost:4011/
<pre>foobar</pre>

Sometimes I can really obsess over very tiny, immaterial details. Such is my burden.

Tomorrow, I am definitely looking at something beside fab.stream!


Day #229

The Case of the Extra Stream Data

‹prev | My Chain | next›

I continue my exploration of the upcoming version 0.5 of fab.js. I admit that I am getting a bit bogged down here, but I believe that understanding is slowing spreading. Hopefully tonight will prove to be a break through. Last night I explored the various degenerative cases of fab.stream. That provided some insight. Tonight I take a step back to see if I can apply that knowledge.

I continue to try to get my player_status (fab) app to embed itself inside HTML template apps:

      (route, /^\/player_status/)
        ( PRE )
          ( player_status )
        () // PRE
      () // route

That is a nice, easy-to-read representation of what I want to do. For the /player_status route, the output of the player_status (fab) app should be wrapped in a <pre> tag.

That is actually pretty easy to do—if the player data were static:

function player_status(write) {
  return write("player_status 01")
  ("\n")
  ("player_status 02")
  ("\n")
  ("player_status 03")
  ("\n");
}

Here, player_status is a normal (fab) app, invoked with the write stream for the response. Invoking that stream with a string argument, "player_status 01", sends the data in the response and also returns the downstream listener in case there is more to say (i.e. "\n", "player_status 02", etc.).

With that, I get my expected 3 player statuses inside a <pre> tag:



My problem is that I need to grab the player status from CouchDB via node-couchdb. That means callbacks. Luckily, even in the infant stages of v0.5, fab.js supplies the fab.stream app to "hit the pause button" until the data is sent back.

How does this cause problems? Well, if I write multiple times to the fab.stream, I get output like this:



And HTML that ends with:

<pre>player_status 02</pre></body></html> 
</pre></body></html>player_status 03</pre></body></html> 
</pre></body></html>player_status 01</pre></body></html> 
</pre></body></html>

I get this even with simple versions without the callbacks, just multiple stream writes:

function player_status( write ) {
  return write(function(write) {
    return fab.stream(function(stream) {
      stream(write("player_status 01"));
      stream(write("\n"));
      stream(write("player_status 02"));
      stream(write("\n"));
      stream(write("player_status 03"));
      stream(write("\n"));
      stream(write());degenerative 
    });
  });
}

In all of the use-cases of fab.stream that I considered last night, I assumed that there was no queue—no initial list of data that would always be written after the explicit writes. That assumption is valid—the queue would be a second argument to fab.stream. Here, I am only supplying a single callback to be invoked—no queue.

My mistake was in not recognizing that there is a second way to build up the queue. The other means of building up the queue is through upstream data. Looking at the fab chain, there does not seem to be any upstream data:

      (route, /^\/player_status/)
        ( PRE )
          ( player_status )
        () // PRE
      () // route

The empty argument function calls should close the HTML tag (and the route) with no data, right? This is were I went wrong. The PRE (fab) app, like all HTML apps, actually creates upstream data beyond player_status. Thinking about it, it has to this in order to create the closing </pre> tag. These HTML (fab) apps write to the normal response stream, then reads from the player_status app while supplying that downstream data.

Taking a peak inside the fab.elem (fab) app, we can see where this occurs:

  function elem( name, isVoid ) {
    return function( write, obj ) {
      write = fab.concat( write )
      write = write( "<" + name );
      write = attrs( write )( obj )( ">" );

      if ( isVoid ) return write;

      return function read( arg ) {
        if ( !arguments.length ) return write( "</" + name + ">" );

        write = write.apply( undefined, arguments );
        return read;
      };
    }

If the upstream app (player_status in this case) returns data, then write that data to the response. If the upstream is done with data (as indicated by no arguments), then write the closing tag.

The end result is that every time I stream new data back, these closing tags are coming along for the ride:

<pre>player_status 02</pre></body></html> 
</pre></body></html>player_status 03</pre></body></html> 
</pre></body></html>player_status 01</pre></body></html> 
</pre></body></html>

In the end, I can get away with (fab) apps with multiple streams only if I am sure that there is no upstream data on the way. Dang.

Armed with that knowledge, I do believe that I can resolve my issue, but I will leave that for tomorrow. I need to get some sleep before Ruby DCamp tomorrow.


Day #228