Async and User Input Testing in Dart

‹prev | My Chain | next›

Newsflash: sometimes staring at code for hours is counter-productive.

I am fighting a focus bug in the ICE Code Editor. The problem is that I can reproduce the bug “In Real Life,” but cannot do so in a test. In real life, I make a code update in the editor, hide the code with the “Hide Code” button and wait for the preview frame to be updated. What should happen is that the preview frame has focus so that keyboard controls in games work. In real life, the containing body element has focus.

No matter what I tried last night (and a little the night before), I could not get my test to fail. Despite my best efforts, my tests always see the iframe focused.

At this point, I have a lot of print() statements scattered throughout the code in an attempt to see what things are being called and in what order. With fresh eyes and fresh brain, I finally compared the output of the failing real-life scenario:

[hideCode] focus undefined:1
[Editor#focus()] preview focus undefined:1
[udpatePreview] focus nop undefined:1
THREE.CanvasRenderer 52 Three.js:11928
[preview_frame] document.close() 

And the test scenario:

[content=] gonna focus undefined:1
[content=] done focus undefined:1
[udpatePreview] focus nop undefined:1
[preview_frame] document.close() preview_frame.html:24
1 TEXTAREA undefined:1
[content=] gonna focus undefined:1
[content=] done focus undefined:1
[hideCode] focus undefined:1
[Editor#focus()] preview focus undefined:1
[udpatePreview] focus nop undefined:1
[preview_frame] document.close() preview_frame.html:24
2 IFRAME 

Darn. It.

The test version is going through the content setter in the Editor class to performs its update whereas as the “In Real Life” version relies on text events to update the content. Well, that certainly explains the differences between the real-life bug and not being able to reproduce it in tests—especially since the content setter plays with keyboard focus (to ensure focus after opening a project).

The question then becomes how can I test this bug if I cannot do the usual (in this project) content setting in order to force preview updates? How can I trigger updates in the code editor like real text events? Especially since it is not possible to trigger keyboard events in Dart?

It may not be possible to trigger keyboard events in Dart, but what about TextEvents? There is an easy way to find out—I replace the editor.content setter with a TextEvent with the same data:

    group("hiding code after update", (){
      setUp((){
        editor.ice.focus();

        // editor.content = '<h1>Force Update</h1>';
        document.
          query('#ice').
          dispatchEvent(
            new TextEvent('textInput', data: '<h1>Force Update</h1>')
          );

        helpers.click('button', text: 'Hide Code')
        // forced delay until preview is updated here...
      });
      test("preview has focus", (){ /* ... */ });
    });

With that, I finally have my failing test:

ERROR: Focus hiding code after update preview has focus
  Focus hiding code after update preview has focus: Test setup failed: Expected: 'IFRAME'
    Actual: 'BODY'   Which: is different.
  Expected: IFRAME
    Actual: BODY

Yay!

Wow, do I love me a failing test. Because now my course of action is clear: make it pass.

Well, maybe not 100% clear because there is the obvious caveat that I need to make it pass without breaking anything else. And in this case, this is an extremely important caveat because the most obvious way to make this test pass would be to have the updatePreview() method tell the editor to focus itself immediately after it updates the preview. This would be a very bad thing to do since I just removed it to fix a separate bug. Having just slogged through async code, my brain is in no condition to play dueling test failures.

This is where a little know feature of Dart's unittest comes in very handy. If you change a single test definition from test() to solo_test(), then only that test is run. That is a huge convenience when you have scores of tests. In this case, I would like a quick way to verify that two tests pass—that I am able to fix the test in question while still not breaking the previous test.

It turns out that you can mark more than one test as a solo_test() and each will run. That might set off oxymoron alarms, but I don't care—it is absolutely brilliant for what I need in this case. With the two tests marked solo_test(), I have:

unittest-suite-wait-for-done
ERROR: Focus hiding code after update preview has focus
  Focus hiding code after update preview has focus: Test setup failed: Expected: 'IFRAME'
    Actual: 'BODY'   Which: is different.
  Expected: IFRAME

PASS: New Project Dialog after preview is rendered input field retains focus if nothing else happens

1 PASSED, 0 FAILED, 1 ERRORS 

If I restore a focus() call in updatePreview(), then I fix one, but break the other:

unittest-suite-wait-for-done
PASS: Focus hiding code after update preview has focus
FAIL: New Project Dialog after preview is rendered input field retains focus if nothing else happens
  Expected: InputElement:<input>
    Actual: TextAreaElement:<textarea>

1 PASSED, 1 FAILED, 0 ERRORS

The actual fix is rather anticlimatic after all of that effort. But the two tests in question make it painfully clear what I need to do. If the context decides when it is appropriate to hide the preview frame, then the calling context needs to be responsible. So, in the UI's hideCode() method that is called when the “Hide Code” button is clicked, I listen for a preview change event and then tell ICE to focus itself:

  void hideCode() {
    ice.hideCode();
    // hide code control UI elements here...

    ice.onPreviewChange.listen((e) {
      ice.focus();
    });
  }

With that, I have both my tests passing:

unittest-suite-wait-for-done
PASS: Focus hiding code after update preview has focus
PASS: New Project Dialog after preview is rendered input field retains focus if nothing else happens

All 2 tests passed.
unittest-suite-success

After moving the two solo_test() functions back to plain old test() functions, I have 116 passing tests and no failures.

Tonight was a big win for me. I have struggled with this focus feature for a while now. My original implementation missed two important edge cases that turned out to be difficult to understand. Difficult to see with all the async flying about, but very worthwhile to be able to test to prevent future regressions. The biggest win of all may be the TextEvent find. It may not work with 100% of my test cases that mimic user input, but it is a significant improvement for simulating user input over simply setting value attributes.


Day #790