perf: Shrink `Table` by `Box`ing the `colliders` and `emptied_collider_slots` in `PointerMap`

[Centril]

This shaves off 40 bytes from PointerMap and in turn from Table. Boxing here is not a problem as collisions are rare.

The perf impact of this hasn't been measured, but large types in general impose memcpys instead of using registers. We should fix this once the more glaring issues have been resolved.

[bfops]

we need to investigate further; we will keep this low priority until we confirm we want to raise it