Use some sort of ray collision detection

[catmanjan]

The problem with rectangle collisions is that they tend to appear to pass through objects, this is more obvious with debug turned on, but even with it off it gives a sense of oddness.

If instead of projecting a box ahead of the mouse velocity, a line was marched in that direction (ala raycasting) I think the experience would be improved.