Arcadia
Arcadia copied to clipboard
arcadia-unity
Testing Framework
There's no coherent way to test Arcadia at the moment, and there should be. Ideally a testing framework should include tests for
- Arcadia's internal machinery
- Unity integration
- Runtime and edit-time features
- Export
We already have clojure.test, I'm not sure we need a big framework aside from that. What would be nice is some tools for running clojure.test, and maybe a bit more knowledge/libraries about how to test the scene graph, especially transitions between edit mode and play mode.
Hm well maybe that's like a testing framework, but we should build it on top of clojure.test rather than haring off to roll our own fundamental test thing.
Weren't you working on a test.check port? How's that coming along?
If we do want to have something that can examine the transition between edit and play mode and automate builds and such, we probably want some external (outside of Unity) system to start things up and shut them down. But that might not be a big deal. Clojure-JVM could easily work for that. We just need to figure out what kind of thing we want to test there.
If this is important I don't think we should block on getting a wonderful multiplatform implementation of Clojure-CLR running externally. We're spread out enough already.
Hm well maybe that's like a testing framework, but we should build it on top of clojure.test rather than haring off to roll our own fundamental test thing.
Agreed. No thoughts here of creating a new testing framework at that level, only of catering to different requirements of testing inside of Unity. If we want to test that e.g. hooks fire properly, we actually need to be in a running game.
Weren't you working on a test.check port? How's that coming along?
It's coming along well. And I started using it to test MAGIC. I am always a bit confused about how to release a port like that, to be honest. The right thing to do seems to be to merge it into the official repo, but then we're waiting on merger in order to use it in our package manager. The other option is to publish it under a different name and be able to use it right away, but that feels like it starts to fragment the ecosystem. Thoughts?
If this is important I don't think we should block on getting a wonderful multiplatform implementation of Clojure-CLR running externally. We're spread out enough already.
No need to block, nostrand already does this. I worry about ClojureJVM external tooling because if we ever need to invoke any CLR code or reference CLR types, we're hosed.
The command line arguments are great, and set us up to start thinking of an automatable testing workflow. For runtime and export tests, I was thinking something along the lines of http://acid3.acidtests.org/, a Unity scene we can build and export, that should be able to run to completion and reach a predetermined state.
The right thing to do seems to be to merge it into the official repo, but then we're waiting on merger in order to use it in our package manager. The other option is to publish it under a different name and be able to use it right away, but that feels like it starts to fragment the ecosystem.
Yes this is a big problem. My strategy has been to fork the repo into a completely new, defensively hideous namespace, such timsg.ports.loom, found here, for loom; then replicate a subset of the functionality of the original library as faithfully as I can, leaving any extensions for an auxiliary library (ie timsg.loom-tools, here). That way if the main branch does get ported it should be easy for users to switch over, and in the meantime we don't have to wait for a complete port to use some essential functionality.
Yeah. That really sucks, but I can't think of an alternative, especially with contrib libraries.
Small issue I ran into that might fall under this:
clojure.test reports using println. Normally these end up in your repl but under Arcadia they show up in the Editor.log making it difficult to actually see what's failing or throwing errors in a test suite.
If there's a workaround it could have a small write up in the wiki.
We have a decent first cut of this in arcadia.internal.test, adequate for internal purposes. Asynchronous, data-oriented, works nicely with the scene graph. Next step is to promote it to a user-facing system.