Add benchmarks and profiling for each config setup

[tonyhb]

We need an idea of benchmarking, latency, and memory usage for each config setup. When building our test rig, it would be great if we could also extend the test rig to provide these metrics.