Add Nethack Benchmarking Script

[aholyoke]

Setup

Download https://alt.org/nethack/xlogfile.full.txt Run this script like: python mk-nethack.py --generate --benchmark --name <name> /path/to/xlogfile.full.txt

--generate Creates a new tdb from the nethack log named <name>.tdb (Not included in benchmark timing) The input can be passed as a file path or piped in from stdin.

eg. head -1000 /path/to/xlogfile.full.txt | python mk-nethack.py --generate --name sample

--benchmark Runs the benchmark on <name>.tdb

The Benchmark

The nethack data has a bunch of causes of death in the death field. eg,

killed by invisible Ms. Possogroenoe, the shopkeeper
slipped while mounting a saddled pony called Argenti
petrified by a cockatrice, while helpless

We take the first word (Usually a verb) and track the average number of points for that death.

The Results

On the full nethack data, this takes about 2 minutes 20 seconds on master and about 48s using the #21 perf-improvements branch.

[Noeda]

Hey, I'm a kind of thinking maybe it would make sense to have a separate repository for example TrailDBs. The TrailDBs themselves aren't tied to any particular traildb bindings, are they?

[aholyoke]

This is a good idea. We should make a repository of example traildbs along with the script that generated them and benchmarks.

[tuulos]

a separate repo of example traildbs is a great idea!