Stronger tests

[cben]

Yesterday I pushed ad146e1 which updated CodeMirror but not Firepad, resulting in a combination where firepad doesn't sync any changes ("pos exceeds the bounds of the AnnotationList").
I knew all about it and even prepared a correct update on a new-deps branch but forgot about it, and only tested the CM commit in my dirty checkout which also had new firepad... FAIL! I should fire myself.

But my main point here is that my laughable smoke test didn't catch that I completely broke mathdown (pages did load but edits weren't saved! That's even worse that being visibly down!)

I really need a functional test opening same page from 2 instances and seeing edits propogate.
Should check if I can reuse existing firepad tests...

Also, JS console errors should be blockers. See https://code.google.com/p/selenium/issues/detail?id=148

[cben]

Jasmine and/or Sauce Labs are giving me grief :-(
In at least half tests I run, Sauce Labs says it didn't receive ANY commands after creating the browser.
Eventually Sauce Labs times out, apparently without informing the client which waits for its own timeout (?).
Jasmine on its part fails with a completely uninformative error:

Started
Creating tunnel...
HttpServer up, e.g. http://localhost:8001/?doc=demo
tunnel created
 > CALL: init({"browserName":"firefox","version":"30.0","platform":"Linux","build":"undefined [undefined] commit undefined","tags":[],"idle-timeout":30,"name":"smoke test","tunnel-identifier":"Tunnel1443696996366"})

Driving the web on session: https://saucelabs.com/tests/24715d5df7324a2da4ea83c95b55efdd

 > RESPONSE: init({"browserName":"firefox","version":"30.0","platform":"Linux","build":"undefined [undefined] commit undefined","tags":[],"idle-timeout":30,"name":"smoke test","tunnel-identifier":"Tunnel1443696996366"})
 > CALL: get("http://localhost:8001?doc=_mathdown_test_smoke")
F > CALL: sauceJobStatus(false)
 > RESPONSE: sauceJobStatus(false)
 > CALL: quit()
 > CALL: init({"browserName":"chrome","version":"30.0","platform":"Linux","build":"undefined [undefined] commit undefined","tags":[],"idle-timeout":30,"name":"smoke test","tunnel-identifier":"Tunnel1443696996366"})

Driving the web on session: https://saucelabs.com/tests/08b765ea48af4f6692c3aa138b7787ab

 > RESPONSE: init({"browserName":"chrome","version":"30.0","platform":"Linux","build":"undefined [undefined] commit undefined","tags":[],"idle-timeout":30,"name":"smoke test","tunnel-identifier":"Tunnel1443696996366"})
 > CALL: get("http://localhost:8001?doc=_mathdown_test_smoke")
F > CALL: sauceJobStatus(false)
 > RESPONSE: sauceJobStatus(false)
 > CALL: quit()
 > CALL: init({"browserName":"safari","version":"8.1","platform":"OS X 10.11","build":"undefined [undefined] commit undefined","tags":[],"idle-timeout":30,"name":"smoke test","tunnel-identifier":"Tunnel1443696996366"})
 > CALL: waitFor({},10000)
 > CALL: eval("document.title.match(/smoke test/)")

Ending your web drivage..


Driving the web on session: https://saucelabs.com/tests/45335742f6444c6bb3b3cec124c1e2f7

 > RESPONSE: init({"browserName":"safari","version":"8.1","platform":"OS X 10.11","build":"undefined [undefined] commit undefined","tags":[],"idle-timeout":30,"name":"smoke test","tunnel-identifier":"Tunnel1443696996366"})
 > CALL: get("http://localhost:8001?doc=_mathdown_test_smoke")
[2015-10-01T11:00:46.089Z] GET localhost:8001 /?doc=... < 127.0.0.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.32 (KHTML, like Gecko) Version/8.1 Safari/601.1.32
[2015-10-01T11:00:46.656Z] GET localhost:8001 /CodeMirror/lib/codemirror.js < 127.0.0.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.32 (KHTML, like Gecko) Version/8.1 Safari/601.1.32
...SNIP...
[2015-10-01T11:00:51.764Z] GET localhost:8001 /favicon.ico < 127.0.0.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.32 (KHTML, like Gecko) Version/8.1 Safari/601.1.32
 > RESPONSE: get("http://localhost:8001?doc=_mathdown_test_smoke")
 > CALL: waitFor({},10000)
 > CALL: eval("document.title.match(/smoke test/)")
 > RESPONSE: eval("document.title.match(/smoke test/)")
 > CALL: eval("document.title.match(/smoke test/)")
 > RESPONSE: eval("document.title.match(/smoke test/)")
 > CALL: eval("document.title.match(/smoke test/)")
 > RESPONSE: eval("document.title.match(/smoke test/)")
 > CALL: eval("document.title.match(/smoke test/)")
 > RESPONSE: eval("document.title.match(/smoke test/)")
 > CALL: eval("document.title.match(/smoke test/)")
 > RESPONSE: eval("document.title.match(/smoke test/)")
 > CALL: eval("document.title.match(/smoke test/)")
 > RESPONSE: eval("document.title.match(/smoke test/)")
 > CALL: eval("document.title.match(/smoke test/)")
 > RESPONSE: eval("document.title.match(/smoke test/)")
 > RESPONSE: waitFor({},10000)
 > CALL: waitForElementByCss(".MathJax_Display",15000)
 > CALL: elements("css selector",".MathJax_Display")
 > RESPONSE: elements("css selector",".MathJax_Display")
 > RESPONSE: waitForElementByCss(".MathJax_Display",15000)
 > CALL: element.text()
 > RESPONSE: element.text()
el.text: α

Error
  at Object.getJasmineRequireObj.errors (/home/beni/mathdown/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:2536:33)
  at Object.getJasmineRequire.core (/home/beni/mathdown/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:45:26)
  at Object.module.exports [as boot] (/home/beni/mathdown/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/node_boot.js:24:32)
  at new Jasmine (/home/beni/mathdown/node_modules/jasmine/lib/jasmine.js:15:30)
  at Object.<anonymous> (/home/beni/mathdown/node_modules/jasmine/bin/jasmine.js:7:15)
  at Module._compile (module.js:456:26)
  at Object.Module._extensions..js (module.js:474:10)
  at Module.load (module.js:356:32)
  at Function.Module._load (module.js:312:12)
  at Function.Module.runMain (module.js:497:10)

(el.text: α above is not part of the error message, it's a debugging print.)

A side effect is that Jasmine crashes a lot, leading to not killing the tunnel, leading to false problems on next run.

[cben]

(as mentioned in afc235c, I found the bug — missing done() — but only after rewriting for Mocha, which clearly reported the issue. I liked Mocha more than Jasmine so switched to it.)