Gnuplot Dart Benchmarks

I am embracing the idea of benchmarking the solutions that I add to Design Patterns in Dart. This early in the research, I am unsure if I will have a strong need for benchmarks. What I do know is that if I cannot make benchmarking easy, then I will almost certainly not use them.

So tonight, I take a quick look at the last piece of my puzzle: visualization. After last night, I have a solution that can store the average run time of a single benchmark run against the number of loops to gather that data. The CSV output looks something like:

Loop_Size, Classic_Visitor_Pattern, Nodes_iterate_w/_single_dispatch, Visitor_Traverses
10, 122.4, 110.6, 117.5, 
100, 118.5, 112.4, 117.6, 
1000, 120.4, 112.4, 116.8, 
10000, 148.6, 148.2, 117.9, 
100000, 148.1, 148.7, 118.5, 

I can paste that data in a spreasheet to produce nice looking graphs like:



But let's face it, if I have to open a spreadsheet every time that I want to look at my data, I am never going to do it. So unless I can find some way to quickly visualize that data, all of this would be for naught. Enter gnuplot.

The preferred data format for gnuplot seems to be tab separated values, so I rework the row output in my Dart benchmark reporter to output tabs instead of commas:

    // ...
    var row = '${loopSize}\t';
    implementations.forEach((implementation){
      var rec = records.firstWhere((r)=> r['name'] == implementation);
      row += '${rec["averageScore"]}\t';
    });
    // ...

The graph type that I want in this case is a clustered histogram culled from data. In gnuplot parlance, that translates as:

set style data histogram
set style histogram clustered

I fiddle a bit with styles (and cull a few from Google / Stack Overflow) to settle on these settings:

set style fill solid border
set xtics rotate out
set key tmargin

The fill style makes the bars a little easier to see and the other two make the data and legends easier to see. Finally, I plot the data as:

plot for [COL=2:4] 'bar.tsv' using COL:xticlabels(1) title columnheader

Which results in:



That is quite nice for just a little bit of work. I do, however, note that this graph suffers from a pretty serious flaw—the same flaw from which my spreadsheet graph suffered unnoticed by me until now. The y-axis does not start at zero, which greatly exaggerates the discrepancies between my three implementations. To start the y-axis at zero, I use:

set yrange [0:]

Now my graph looks like:



Now that I see the “jump” in values at the 10k loop size in that perspective, it does not seem nearly as worrisome.

If I put that all in a gnuplot script file:

# set terminal png size 1024,768
# set output  "bar.png"
set style data histogram
set style histogram clustered
set style fill solid border
set xtics rotate out
set key tmargin
set yrange [0:]
plot for [COL=2:4] 'bar.tsv' using COL:xticlabels(1) title columnheader
pause -1 "Hit any key to continue"

Then I can quickly see my benchmarking results whenever I need to. No excuses.


Day #125

‹prev | My Chain | next›