Stop boxing primitives

[kazimuth]

If we use something like high-performance primitive collections we may be able to get a free performance boost.

It's a pretty big dependency, though, so we probably want some way of getting rid of all the stuff we're not using. ProGuard does that, I think?