Showing posts with label selenium. Show all posts
Showing posts with label selenium. Show all posts

Sunday, December 14, 2014

Trying Protractor to Test Polymer


Well, if I've gone to all of the trouble of getting Selenium installed...

Despite not being able to get The Intern to properly test Polymer elements, the effort was not a complete loss. There are some nice features in Intern that I wish existed elsewhere in JavaScript testing space. My favorite is probably the built-in web server. I know it's stupid, but I hate having to run my own web server in tests—there are just so many other moving parts that adding another is a source of some consternation. I also picked up a useful little trick from the Intern's configuration. They suggest an NPM post-install script to Bower install client side packages:
{
  "name": "plain_old_forms",
  // ...
  "scripts": {
    "postinstall": "bower install",
    "test": "intern-runner config=tests/intern"
  }
}
That seems perfectly reasonable, to the point of a best practice kind of thing, so I will likely adopt it going forward.

That said, my experience with the Intern was ultimately a failure since it was not able to reference Polymer elements, which puts a damper on testing them. But since I went to all the trouble of getting all things Selenium and Web Driver installed yesterday, I think that I will give Protractor a shot with testing Polymer elements. Sure, Protractor is really meant for testing AngularJS code and it requires a separate web server to test code, but it installs a bunch of the WebDriver stuff for me and, who knows, maybe it'll actually be able to reference Polymer elements.

I have fiddled with Protractor before, but it has been a while:
$ protractor --version
Version 0.22.0
So I upgrade:
$ npm install -g protractor
/home/chris/local/node-v0.10.20/bin/protractor -> /home/chris/local/node-v0.10.20/lib/node_modules/protractor/bin/protractor
/home/chris/local/node-v0.10.20/bin/webdriver-manager -> /home/chris/local/node-v0.10.20/lib/node_modules/protractor/bin/webdriver-manager
protractor@1.5.0 /home/chris/local/node-v0.10.20/lib/node_modules/protractor
...
➜  protractor git:(master) ✗ protractor --version     
Version 1.5.0
I already have the latest Selenium installed, but Protractor prefers Chrome as the default browser testing environment, which I do not have. So I let Protractor install it:
$ webdriver-manager update
Updating selenium standalone
...
Updating chromedriver
...

I have to admit that, even though it is a thin wrapper around the normal Java Selenium server startup command, I find Protractor's thin wrapper much more appealing:
webdriver-manager start
seleniumProcess.pid: 26247
23:40:45.524 INFO - Launching a standalone server
Setting system property webdriver.chrome.driver to /home/chris/local/node-v0.10.20/lib/node_modules/protractor/selenium/chromedriver
23:40:45.570 INFO - Java: Oracle Corporation 24.65-b04
23:40:45.571 INFO - OS: Linux 3.13.0-37-generic amd64
23:40:45.585 INFO - v2.44.0, with Core v2.44.0. Built from revision 76d78cf
explorer, version=}] does not match with current platform: LINUX
23:40:45.734 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
...
23:40:45.766 INFO - Started HttpContext[/wd,/wd]
23:40:45.770 INFO - Started SocketListener on 0.0.0.0:4444
23:40:45.770 INFO - Started org.openqa.jetty.jetty.Server

I already have a smoke test page for the <x-pizza> pizza building element that I would like to test, so I fire up a Python simple server against which my tests can run:
$ python -m SimpleHTTPServer 8000
Serving HTTP on 0.0.0.0 port 8000 ...

This is a fresh copy of my <x-pizza> element. It has a bower.json for all of the necessary dependencies as well as an npm package.json for helper packages like Grunt. I add the post-install "trick" from the Intern to my package.json:
{
  "name": "plain_old_forms",
  "devDependencies": {
    "grunt": "~0.4.0",
    "grunt-contrib-watch": "~0.5.0"
  },
  "scripts": {
    "postinstall": "bower install"
  }
}
After an npm install, I am ready to test:
describe('<x-pizza>', function(){
  var el;

  describe('element content', function(){
    beforeEach(function(){
      browser.get('http://localhost:8000');
    });
    it('has a shadow DOM', function(){
      el = $('x-pizza');
      expect(el.shadowRoot).not.toBe(null);
    });
  });
});
This is not a "real" test. It just verifies that the <x-pizza> element is a real Polymer element—that it has a shadow root. This is the test that I was never able to get running on the Intern.

Unfortunately, this does not work any better in Protractor than in Intern. When I run the test, I get:
protractor conf.js
Using the selenium server at http://localhost:4444/wd/hub
[launcher] Running 1 instances of WebDriver
F

Failures:

  1) <x-pizza> element content has a shadow DOM
   Message:
     Error: Angular could not be found on the page http://localhost:8000/ : retries looking for angular exceeded
   Stacktrace:
     Error: Angular could not be found on the page http://localhost:8000/ : retries looking for angular exceeded
==== async task ====
Protractor.get(http://localhost:8000/) - test for angular
    at [object Object].<anonymous> (/home/chris/repos/polymer-book/play/protractor/tests/XPizzaSpec.js:6:15)
==== async task ====
Asynchronous test function: beforeEach()
Error
    at [object Object].<anonymous> (/home/chris/repos/polymer-book/play/protractor/tests/XPizzaSpec.js:5:5)
    at [object Object].<anonymous> (/home/chris/repos/polymer-book/play/protractor/tests/XPizzaSpec.js:4:3)
    at Object.<anonymous> (/home/chris/repos/polymer-book/play/protractor/tests/XPizzaSpec.js:1:63)

Finished in 10.667 seconds
1 test, 2 assertions, 1 failure
So it seems that Protractor is more than a little coupled to Angular. Bummer.

I am somewhat surprised that Protractor is coupled to Angular to the point that it does not work with anything else. I had just assumed that it worked better with Angular than with other things. That is good to know, I suppose, but not terribly helpful for my current efforts. I may give the basic selenium-webdriver NPM package a try tomorrow, but that may be veering too close to my dislike of raw Selenium.


Day #24

Saturday, December 13, 2014

Can't Even Polymer with Intern and Local Selenium Server


I hate a problem unsolved. I really ought to move onto more useful ground, but I cannot for the life of me understand why I cannot get The Intern tester running with Polymer elements. So tonight, I try to remove one potential problem point from the equation: Sauce Labs.

Really, there is no reason that this should work, but if nothing else, running locally will give me more real time feedback on what might possibly be going on here. But I'll be honest here, I hate Selenium locally (which is one of the reasons I really appreciate Sauce Labs).

Running it is easy enough. I download the latest standalone server from the download page. I can then run the server immediately with:
$ java -jar Downloads/selenium-server-standalone-2.44.0.jar
00:07:06.282 INFO - Launching a standalone server
00:07:06.366 INFO - Java: Oracle Corporation 24.65-b04
00:07:06.366 INFO - OS: Linux 3.13.0-37-generic amd64
00:07:06.388 INFO - v2.44.0, with Core v2.44.0. Built from revision 76d78cf
...
00:07:06.606 INFO - Started HttpContext[/wd,/wd]
00:07:06.611 INFO - Started SocketListener on 0.0.0.0:4444
But for the life of me, I cannot figure out how to install the Firefox driver. How stupid is that? This must be so obvious to just about everyone else, but I've got nothing.

Eventually, I read that the extension is contained in the standalone server. I would think that would mean that I could install it from the running server, but if that is case, I am unable to make it happen. So I unzip the JAR file:
$ unzip ../selenium-server-standalone-2.44.0.jar
...
   creating: org/openqa/grid/selenium/
   creating: org/openqa/grid/selenium/proxy/
   creating: org/openqa/grid/selenium/utils/
  inflating: org/openqa/grid/selenium/GridLauncher.class
  inflating: org/openqa/grid/selenium/proxy/DefaultRemoteProxy$1.class
  inflating: org/openqa/grid/selenium/proxy/DefaultRemoteProxy.class
  inflating: org/openqa/grid/selenium/utils/WebProxyHtmlRenderer.class
That does contain the XPI:
$ find | grep xpi$
./org/openqa/selenium/firefox/webdriver.xpi
Armed with that, I install it by directly pointing Firefox to the location on disk:



After allowing this to occur:



I am ready to make some changes to the Intern tests/intern.js configuration code. I switch from a specific browser version on Sauce Labs to my local Firefox:
  environments: [
  // { browserName: 'chrome', version: '34', platform: [ 'Linux' ] }
  { browserName: 'firefox' }
  ]
I switch from the Sauce Labs tunnel to no tunnel:
  // Name of the tunnel class to use for WebDriver tests
  //tunnel: 'SauceLabsTunnel',
  tunnel: 'NullTunnel',
  useSauceConnect: false,
Finally, I add webdriver configuration (this may not be necessary):
  webdriver: {
    host: 'localhost',
    port: 4444
  },
With that in place, I am ready to try again… which results in:
$ ./node_modules/.bin/intern-runner config=tests/intern
Listening on 0.0.0.0:9000
Starting tunnel...
Initialised firefox 34.0 on LINUX
Test main - index - <x-pizza> FAILED on firefox 34.0 on LINUX:
CancelError: Timeout reached on main - index - <x-pizza>
Error: Timeout reached on main - index - <x-pizza>
  at Error  <anonymous>
  at new ErrorCtor  <node_modules/intern/node_modules/dojo/errors/create.js:13:21>
  at null._onTimeout  <node_modules/intern/lib/Test.js:180:38>
  at Timer.listOnTimeout [as ontimeout]  <timers.js:110:15>
Unfortunately, this seems even worse than yesterday. The timeout is occurring while waiting for the Polymer element's shadow root property to come into existence:
    '<x-pizza>': function () {
      return this.remote
        .get(require.toUrl('../index.html'))
        .setFindTimeout(5000)
        .findByXpath('//body[not(@unresolved)]')
        .then(pollUntil('return document.querySelector("x-pizza").shadowRoot;', 1000))
        .findByTagName('x-pizza')
        // .execute('return document.querySelector("x-pizza");')
        .then(function (el) {
           console.log(el);
           console.log(el.$);
           console.log(el.constructor);
           console.log(el.shadowRoot);
           console.log(el[0]);
           assert.isDefined(el.shadowRoot);
         });
    }
That at least happened with Sauce Labs, but never returns locally. This is baffling because I can see the <x-pizza> element just starting to be resolved after Polymer has initialized and removed the unresolved attribute from the <body>:



Unfortunately, I cannot think of anything else to try tonight that I had not already tried last night with Sauce Labs. If nothing else, I can see very definitely tonight that the <x-pizza> element is present and should have a shadowRoot (which is where the SVG pizza resides). But for whatever reason, the Intern is unable to access this element for testing.

I think I am ready to give up on Intern for real now, but I may give Protractor a whirl with this tomorrow. Even if this particular WebDriver tool cannot yet test Polymer, perhaps another tool can.


Day #23