Automated, Full Stack Testing in Dart

‹prev | My Chain | next›

Yesterday, I made my first legitimate foray into testing full stack Dart web applications. I am testing the data persistence layer in the Hipster MVC library (which is an outgrowth of the book Dart for Hipsters). One of the nice things about Hipster MVC is that the persistence layer, HipsterSync, can swap out behaviors so that persistence can reside in browser localStore just as easily as it does in a RESTful web service. That swap-ability makes testing applications based on Hipster MVC nice, but the default HTTP persistence needs to be tested somehow...

As of last night, I have a simple test that verifies that HipsterSync can retrieve data from a live web service:

      test("it can parse responses", (){
        _test(response) {
          expect(response, {'foo': 1});
        }

        var model = new FakeModel();
        HipsterSync.call('get', model).then(expectAsync1(_test));
      });

For that to work, I need a test server that always returns the JSON string {'foo': 1}. I wrote that server yesterday and the test passes, but I need to start that server before I run that test:

➜  test git:(master) ✗ dart test_server.dart
Server started on port: 31337

There are a number of concerns about this, but the one that I choose to focus on tonight is: how can I possibly run this under continuous integration?

A Dart web server relies on the dart:io package to do its thing:

import 'dart:io';
import 'dart:json' as JSON;

main() {
  HttpServer.bind('127.0.0.1', port).then((app) {
    // ...
    print('Server started on port: ${port}');
  });
}

The problem as far as testing is concerned is that the dart:io library is not available in a browser context—it is strictly for use in the server side Dart VM. But, since HipsterSync relies on the dart:html library, HipsterSync can only run (and be tested) in a browser context. The end result is that I cannot start a web server from a test that is exercising HipsterSync. Again, there may still be other options for me, but for now I continue to focus on just running this test under continuous integration.

Dart tests for browser code uses the content_shell utility for browser context on the command line. For Hipster MVC, this is done as a Bash script, which works just fine locally and on the very excellent drone.io service. So let's see if I can fork the test server process, run my tests and kill the server:

echo "starting test server"
dart test/test_server.dart &
server_pid=$!

echo "content_shell --dump-render-tree test/index.html"
results=`content_shell --dump-render-tree test/index.html 2>&1`
# Check results here...

kill $server_pid

# Run dartanalyzer checks here...

And, when I run that script it works:

➜  hipster-mvc git:(master) ✗ ./test/run.sh
starting test server
content_shell --dump-render-tree test/index.html
Server started on port: 31337
CONSOLE MESSAGE: unittest-suite-wait-for-done
CONSOLE MESSAGE: PASS: unsupported remove
CONSOLE MESSAGE: PASS: Hipster Sync can parse regular JSON
CONSOLE MESSAGE: PASS: Hipster Sync can parse empty responses
CONSOLE MESSAGE: PASS: Hipster Sync HTTP get it can parse responses
CONSOLE MESSAGE: 
CONSOLE MESSAGE: All 4 tests passed.
CONSOLE MESSAGE: unittest-suite-success

The big question is, will this work on drone.io? I am very carefully binding only to localhost (127.0.0.1) in the test_server.dart script and am definitely binding to a non-root port (i.e. >1024). Even if I were not doing both of those things, it might work since the builds are in a sandbox with sudo privileges (though that may be limited to xvfb support). But if it is going to work, then this localhost:31337 host:port approach should do it.

And it does. The build passes and the test output indicates that the test server is started and stopped as expected.

My current test server is fairly useless—only returning the JSON string {'foo': 1} no matter the request or HTTP verb. I think that I will try a slightly more sophisticated test server tomorrow. Regardless, I still want to be able to stub out HTTP requests in Dart for testing API boundaries and the like. From what I have seen so far, this is not possible, but I still hold out hope for something useful.


Day #803