Monday, June 30, 2014

Benchmarking Dart (and dart2js) Code in the Browser


I am a benchmarking fool. I so enjoyed benchmarking various implementations of the Factory Method pattern, that I try it again today. Yesterday I ran the benchmarks from the command-line. Today, I wonder what my options are for benchmarking Dart code in the browser.

To benchmark Dart from the command-line, I used the dart system command that comes with the SDK. For good measure, I also compiled my benchmark harness into JavaScript and ran it with node.js. Amazingly, that worked without any effort on my part.

It worked so well, I thought why not try it in the browser? There may wind up being a web-specific design pattern or two in Design Patterns in Dart, so while this benchmarking stuff is fresh in my brain, I ought to see how it might work in the browser instead of from the command-line. With a qualifier...

Even though these benchmarks will be run in the browser, I want the results readily accessible from the command-line. I would like the ability to review progress of any benchmarks as I refine pattern approaches and that needs to be automated or it simply won't happen.

So I create a web sub-directory in my factory_method sample code, copy in yesterday's benchmark.dart and start with a simple index.html:
<!DOCTYPE html>
<html>
  <head>
    <script type="application/dart" src="benchmark.dart"></script>
    <script src="packages/browser/dart.js"></script>
  </head>
  <body></body>
</html>
I do not know how these things are normally done in JavaScript, but if I want the console.log output from Dart code, I stick with content_shell, which is bundled with the SDK. And this works right off the bat:
$ content_shell --dump-render-tree web/index.html
#READY
CONSOLE MESSAGE: Factory Method — Subclass(RunTime): 0.09106837778077291 us.
CONSOLE MESSAGE: Factory Method — Map of Factories(RunTime): 1.8514472763359118 us.
CONSOLE MESSAGE: Factory Method — Mirrors(RunTime): 12.328632014991616 us.
Content-Type: text/plain
layer at (0,0) size 800x600
  RenderView at (0,0) size 800x600
layer at (0,0) size 800x8
  RenderBlock {HTML} at (0,0) size 800x8
    RenderBody {BODY} at (8,8) size 784x0
#EOF
#EOF
#EOF
Nice. Those numbers are comparable to the Dart command-line numbers from yesterday, so all appears to be in good shape with benchmarking pure Dart in the browser. What about dart2js compiled JavaScript?

I could try dumping it into a test runner like Karma, but that seems crazy. I mean more crazy than most of the stuff I try. Maybe content_shell will work for this as well? To test that out, I pub build my benchmark application which compiles the index.html page and associated code:
$ pub build
Loading source assets...
Building factory_method_code...
[Info from Dart2JS]:
Compiling factory_method_code|web/benchmark.dart...
[Dart2JS on factory_method_code|web/benchmark.dart]:
1 warning(s) suppressed in dart:_js_mirrors.
[Warning from Dart2JS]:
web/benchmark.dart:
2391 methods retained for use by dart:mirrors out of 3411 total methods (70%).
[Info from Dart2JS on factory_method_code|web/benchmark.dart]:
packages/factory_method_code/mirrors.dart:3:1:
This import is not annotated with @MirrorsUsed, which may lead to unnecessarily large generated code.
Try adding '@MirrorsUsed(...)' as described at https://goo.gl/Akrrog.
import 'dart:mirrors';
^^^^^^^^^^^^^^^^^^^^^^
[Info from Dart2JS]:
Took 0:00:13.648601 to compile factory_method_code|web/benchmark.dart.
Built 5 files to "build".
Note to self: I really need to look into that @MirrorUsed annotation. Some other day.

For now, I have compiled JavaScript and page in the build directory:
$ tree -L 2 build
build
└── web
    ├── benchmark.dart.js
    ├── benchmark.dart.precompiled.js
    ├── index.html
    └── packages

2 directories, 3 files
I cannot just run that code because content_shell from the Dart SDK has the Dart VM included. With the Dart VM available, content_shell would try to run the Dart code which was not included in the build process.

So, just to see if this works, I hand-edit the generated HTML to directly point to the compiled benchmark.dart.js file instead of relying on the usual browser/dart.js to do it:
<!DOCTYPE html>
<html>
  <head>
    <!-- <script type="application/dart" src="benchmark.dart"></script> -->
    <!-- <script src="packages/browser/dart.js"></script> -->
    <script src="benchmark.dart.js"></script>
  </head>
  <body></body>
</html>
With that, I can run content_shell against build/web/index.html to find:
content_shell --dump-render-tree build/web/index.html
CONSOLE MESSAGE: line 14349: Factory Method — Subclass(RunTime): 0.46883995866706923 us.
CONSOLE MESSAGE: line 14349: Factory Method — Map of Factories(RunTime): 5.1614768017425146 us.
CONSOLE MESSAGE: line 14349: Factory Method — Mirrors(RunTime): 16.81279790176282 us.
Content-Type: text/plain
layer at (0,0) size 800x600
  RenderView at (0,0) size 800x600
layer at (0,0) size 800x8
  RenderBlock {HTML} at (0,0) size 800x8
    RenderBody {BODY} at (8,8) size 784x0
#EOF
#EOF
#EOF
Again, those numbers are comparable to the command-line dart2js results from yesterday which seems to confirm that this will work.

Unless someone cares an awful lot (and tells me that I am mistaken), this seems a reasonable approach to benchmarking in the browser. I may poke around a little more tomorrow—or jump right into adding a pub build transformer to automatically change the <script> tags that I hand edited today.

But so far, this seems promising.


Day #108

No comments:

Post a Comment