Results Appear to be Inconsistent With Baseline

[joinr]

I got around to installing rust and running your branch on a 2-core EC2 m4.large instance running linux on openjdk 1.8-212, and fresh install of rust via rustup . I ran your rust implementation using the

cargo run --release problems/*.desc --threads=2

command. Clojure ran under the same threadcount on the AWS box.

During comparison (aside from the typical performance metrics), I also remembered to check the scores for the rust implementation. I ran into a problem early on using unchecked math in clojure that caused results to diverge, generally getting lower scores on bigger problems (at least prob-277) and obfuscating experimentation. After that, I generated scores.edn from the baseline branch (your original competition submission), and verified output against that.

I did the same for the rust output and found that the output appears inconsistent for all the problems. There isn't a clear trend (e.g. lower scores in general), and I haven't analyzed them all. Problem 277 sticks out since that's what I tested against the most. In this case, the Rust implementation gets a lower score. The same thing happened when I had the unchecked-math bug (I don't currently understand why it happened).

I don't know whether this is chalked up to platform differences or something else, but the Rust implementation does not appear to be achieving the same results as the original clojure implementation (and transitively the optimized branch). Which is "actually" correct, I can't say absent some absolute truth to compare against from the ICFPC.

analysis here