Testing Async Hell: Dart Style

‹prev | My Chain | next›

Man, testing is hard. Actually testing is pretty easy—especially in Dart.

But man is testing async stuff is hard. Dart provides plenty of little tricks to make it easier. Async testing has first-class support in Dart, but just because it is there does not mean that it is easy to setup test cases well.

Last night, Santiago Arias (my #pairwithme partner) and I were able to isolate one problem in the ICE Code Editor. Last night's problem was that something was stealing focus from the new project input field after clicking on it. The baffling thing was that we had a test already in place that was still passing:

    test("new project input field has focus", (){
      helpers.click('button', text: '☰');
      helpers.click('li', text: 'New');

      expect(
        query('.ice-dialog input[type=text]'),
        equals(document.activeElement)
      );
    });

It was a pretty straight-forward test which had served nicely to drive the original implementation. Click the main menu button (the ☰ symbol), then select the “New” menu item. The expectation is that the text field in the resultant dialog has focus. And even though this was not the case in the actual application, this test passed.

Santiago and I eventually reproduced the failure in tests by waiting for the preview to have been updated, and then waiting a single clock tick after clicking the “New” menu item:

    group("after preview is rendered", (){
      setUp((){
        // Return future so that setup is only considered complete when preview
        // had changed
        var preview_ready = new Completer();
        editor.onPreviewChange.listen((e){
          preview_ready.complete();
        });
        return preview_ready.future;
      });

      test("input field retains focus if nothing else happens", (){
        helpers.click('button', text: '☰');
        helpers.click('li', text: 'New');

        Timer.run(expectAsync0((){
          print(document.activeElement.tagName);
          expect(
            document.activeElement,
            equals(query('.ice-dialog input[type=text]'))
          );
        }));
      });
    });

With that, the test would fail with:

FAIL: New Project Dialog after preview is rendered input field retains focus if nothing else happens
  Expected: InputElement:<input>
    Actual: TextAreaElement:<textarea>

This is exactly what the application bug was doing: after clicking the “New” menu item, the textarea in the code editor was active—not the input field in the dialog.

We then found that we could fix this breaking test (and the bug) by disabling the focus() call in the updatePreview() method of the editor:

updatePreview() {
      // ...
      // focus();

      // Trigger an onPreviewChange event:
      _previewChangeController.add(true);
  }

What is odd about this is that focus() call is happening before the onPreviewChange event that kicks off the test.

Anyhow, that is all well and good, but I added that now-commented-out focus() call for a reason. It is responsible for giving focus to the preview element if the code is updated and then hidden with the hide code button. The use case here is that the programmer makes a change and wants to see the code right away. Since this code could very well contain keyboard listeners, focus is vital. Hence the test:

    group("hiding code after update", (){
      setUp((){
        editor.ice.focus();
        editor.content = '<h1>Force Update</h1>';
        helpers.click('button', text: 'Hide Code');

        var preview_ready = new Completer();
        editor.onPreviewChange.listen((e){
          preview_ready.complete();
        });
        return preview_ready.future;
      });

      test("preview has focus", (){
        var el = document.query('iframe');

        expect(document.activeElement, el);
      });
    });

The problem is that I am unable to make this test fail, no matter what delays I add, this test continues to pass:

    group("hiding code after update", (){
      setUp((){
        editor.content = '<h1>Force Update</h1>';
        helpers.click('button', text: 'Hide Code');

        var preview_ready = new Completer();
        editor.onPreviewChange.listen((e){
          preview_ready.complete();
        });
        return preview_ready.future;
      });

      test("preview has focus", (){
        var wait = new Duration(milliseconds: 200);
        new Timer(wait, expectAsync0((){
          new Timer(wait, expectAsync0((){
            expect(document.activeElement.tagName, 'IFRAME');
          }));
        }));
      });

Bother.

Somehow, in the live application, the body of the main page is retaining focus (or it usurping focus at some point after the preview element tried to grab its rightful focus). But try as I might, I have not figured out the magical combination of async setup in my tests that can reproduce this. This is going to bug me all night, but is probably one of those situations in which sleep will reveal something obvious

Regardless, if I cannot reproduce this problem, then I cannot fix it—at least in a meaningful way that will stay fixed.


Day #789