How to deal with error accumulation?

[mrgloom]

How to deal with error accumulation that happens from frame to frame? i.e. suppose we have some ideal bbox but in real life we predict bbox+eps if we make this eps mistake from frame to frame this can lead to bbox expand to whole image or shrink to single pixel.