grid like rpn after 600 iterations

[elhamAm]

I am using a resnet for the backbone instead of a transformer and after 600 iterations the region proposals coming out of the rpn start looking like a grid. Even though in the early iterations the Rpn looks reasonable with reasonable clusters. Any clue as to what is wrong? I make sure that the feature map levels that I take have the same shape as your version.