Compaction/cleanup improvements

[markpapadakis]

Cleanup works fine, but we need to improve it with throttling support, and by exposing some hard coded options as configuration options.

• [ ] Throttling. We are now reading and writing as fast as we can, with no artificial delay between operations. This is not optimal, because it can affect serving consume and produce requests. I also don't think we should rely on some static rate, and instead we should consider the the state of the broker, and make sure we adjust the rate based on it, dynamically.
  If the broker is mostly idle, we should increase the rate, if the broker is busy servicing consume and produce requests, we should lower the rate.
• [ ] Configuration options for maximum bundle size, minimum log file size, etc Those are now hardcoded in compact_partition(), and I think they the default values are pretty reasonable, but we should consider exposing them as configuration option