johnmyleswhite
[PkgEval] Benchmark may have a testing issue on Julia 0.3 (2014-05-19)
PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.2) and the nightly build of the unstable version (0.3). The results of this script are used to generate a package listing enhanced with testing results.
On Julia 0.3
- On 2014-05-17 the testing status was
Tests pass.. - On 2014-05-19 the testing status changed to
Tests fail, but package loads..
Tests pass. means that PackageEvaluator found the tests for your package, executed them, and they all passed.
Tests fail, but package loads. means that PackageEvaluator found the tests for your package, executed them, and they didn't pass. However, trying to load your package with using worked.
This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.
Looks like this is a periodic thing due to a tight tolerance:
ERROR: test failed: maximum(comparisons[:Relative]) < 1.1