Regression Testing of Dart Type Analysis

‹prev | My Chain | next›

I think Dart's killer feature may very well turn out te be its support for testing. If robust, accurate, maintainable code it the desired state for an application, then testing is a must.

Surprising to me, as a old time Ruby and JavaScript hacker, is the ability of static typing to catch bugs that I would have otherwise missed. When Java programmers complained to me about Ruby's lack of typing, I would scoff, kick up my toe-shoed heels, and wax poetic about how strong testing was all that was needed for high quality code. Well, Dart has made me a convert.

As much as I love static typing now, I am completely ignoring it in my drone.io regression tests. So let's add it. In run.sh, I add the analysis below the unit tests:

#!/bin/bash

set -e

#####
# Unit Tests

echo "DumpRenderTree test/index.html"
results=`DumpRenderTree test/index.html 2>&1`

echo "$results" | grep CONSOLE

echo $results | grep 'unittest-suite-success' >/dev/null

echo $results | grep -v 'Exception: Some tests failed.' >/dev/null

#####
# Type Analysis

echo
echo "dart_analyzer lib/*.dart"

results=`dart_analyzer lib/*.dart 2>&1`

echo "$results"

if [ -n "$results" ]; then
    exit 1
else
    echo "Passed analysis."
fi

Unfortunately, dart_analyzer does not return proper Unix exit codes when it detects a problem. That is, even if there is a problem, it still exits with zero exit code. So instead of relying on the set -e in the script to exit on any error, I check to see if the string returned from dart_analyzer is non-zero length with the -n bash operator.

When I intentionally create a static typing error, I now get a non-zero exit status:

➜  hipster-mvc git:(master) ✗ ./test/run.sh
DumpRenderTree test/index.html
CONSOLE MESSAGE: unittest-suite-wait-for-done
CONSOLE MESSAGE: PASS: unsupported remove
CONSOLE MESSAGE: 
CONSOLE MESSAGE: All 1 tests passed.
CONSOLE MESSAGE: unittest-suite-success

dart_analyzer lib/*.dart
file:/home/chris/repos/hipster-mvc/lib/hipster_collection.dart:7:7: Concrete class HipsterCollection has unimplemented member(s) 
    # From HipsterCollection:
        String url
        HipsterModel modelMaker(dynamic)
     6: 
     7: class HipsterCollection implements Collection {
              ~~~~~~~~~~~~~~~~~
➜  hipster-mvc git:(master) ✗ echo $?      
1

And, if I fix the problem (by declaring this class as abstract), I now have a passing test suite, including type analysis:

➜  hipster-mvc git:(master) ✗ ./test/run.sh
DumpRenderTree test/index.html
CONSOLE MESSAGE: unittest-suite-wait-for-done
CONSOLE MESSAGE: PASS: unsupported remove
CONSOLE MESSAGE: 
CONSOLE MESSAGE: All 1 tests passed.
CONSOLE MESSAGE: unittest-suite-success

dart_analyzer lib/*.dart

Passed analysis.
➜  hipster-mvc git:(master) ✗ echo $?      
0

I push this new test to Hipster MVC and call it a day. A green day:

Day #629