Run upstream mosh tests

[rpwoodbu]

I understand that upstream mosh has grown some tests. I should teach the build system how to run them.

[cgull]

As the author of said tests, I applaud the sentiment, but that may be...challenging. The tests are mostly end-to-end tests that inspect the client's behavior when poked and prodded with test programs at both client and server. They use tmux to provide a client terminal emulation that screen dumps can be grabbed from, and really expect the mosh-client program to be a CLI program-- does the build environment provide a CLI, non-browser emulation/execution environment?

Alternately, if you can get mosh-chrome to run in its normal GUI environment and hterm can be convinced to inject keystrokes and do textual screen dumps, that could replace our use of tmux. Various parts of the tests would require some rework to eliminate tmux-specific code though.

You'd also have to provide a mosh-server-- probably your build environment's host platform's packaged version is fine, except then you have issues with testing something that's not actually from your build. Many of the terminal-emulation issues we've had have been server-specific (the diffs from the server look almost exactly like the updates mosh-client sends to the client terminal, and use a relatively small subset of ANSI terminal emulation functionality).

The test framework is a fairly complicated mess of scripts, and so far mostly tests issues related to the fidelity of terminal emulation-- I'd certainly appreciate any suggestions for simplifying/improving it, and I'd be willing to work with you on that.

[rpwoodbu]

Thanks for those details, @cgull. I haven't actually looked at your tests, so I didn't know their nature; I think I'd just assumed they were unit tests.

Let me give you a little background on how Mosh for Chrome is built, and why I thought it'd be good to run your tests. Mosh for Chrome uses Bazel. While it is technically possible to make it run make for building the upstream Mosh, it is actually simpler (and more versatile) to teach Bazel how to build Mosh itself, without using your Makefiles. While it is unlikely that I'd even get a successful compilation/link if Bazel were misconfigured, it would be nice to have the tests as a kind of "belt and suspenders" that I've built the thing correctly.

(If you want a bit more on why I'm using Bazel, see the end of my Code Walk.)

In Bazel, the intent is for unit tests to run on the host platform, so getting mosh-client and mosh-server built as CLI programs is not a problem. Bazel endeavors to be hermetic, so I'd also need to teach it how to build tmux (should be simple).

I don't see any problem with using tmux as the mechanism by which to do end-to-end testing, but I suppose there could be some value in generalizing your scripts so that a variety of terminal emulators could be tested in an effort to discover interop bugs. Using hterm is tempting, since Mosh for Chrome definitely cares about interop with that terminal emulator, but I'm not sure I'd get enough value for the cost, since the hard part would be running a headless Chrome and plumbing all that mess.

I'll look into your tests and see if I can run them in Bazel without too much trouble. If I think of any suggestions for them along the way, I'll be sure to let you know!

[rpwoodbu]

After spending some time on this, I have found that it would take a fair amount of work to get these tests running in Bazel, unfortunately. Bazel is rather opinionated about how files are laid out. It is part of its hermetic and deterministic nature. There are numerous places in your shell scripts where files are expected to be available via $PATH, or with respect to $PWD, etc. It could be cleaned up, but it would just take some time.

But here's the other thing: I can't easily link against openssl. Yeah, I know, crazy, right? For Mosh for Chrome, I am actually getting a pre-built PNaCl archive from this place called "webports" (run by Google). It's kind of a cheat; Bazel would prefer to build from source. But then I'd have to go to the trouble of defining a Bazel BUILD file for openssl. The alternative would be to use a pre-built amd64 archive. That's not the end of the world; there are several small portability violations in my Bazel setup already, although none so overt as that. But I would rather not go down that path. I did happen to find someone who had offered a BUILD file for the openssl project, but he was summarily rejected. I added my plea, but I'm not holding my breath.

So this is probably more trouble than it's worth. I did get to kick the tires on your tests, though. I'm really glad to see this; the hardest part is always writing the first test. You've provided a framework onto which both simple unit tests and complex end-to-end tests can easily be added. I didn't delve too deeply into the code, but I can offer a few high-level observations:

• The end-to-end tests could stand to run faster, if possible. This would be mitigated by the next item, though.
• You should consider breaking out the unit tests from the end-to-end tests, so devs can quickly launch unit tests while they work. I'd recommend making it so that unit tests are nearby the code-under-test, so you can just type make check in the directory where you're already sitting.
• Please add a note to the src/tests/README.md to say to run make check. I actually didn't know that was the automake way of doing things. I was sitting there trying variations of make test, and looking for some master shell script, before I had the good sense to read the automake docs.

I say all this, while my project has next to no tests! I am honestly a bit embarrassed by that, as I really promote testing in my professional life. When I started this project, I was unfamiliar with the NaCl toolchain(s), and cobbled everything together with a nasty shell script and hand-crafted Makefile. Now that I am quite familiar with it all, and have a lovely build system, I should really write tests. But I basically don't touch most of the foundational code at this point.