How about not sharing point-level queries in the one2many assigner

[JingweiZhang12]

@LegendBC @wondervictor @outsidercsy Thanks for your code! I'm curious that sharing point-level queries could cause interaction between one2one group and one2many group in the training, which is not the desired operation in the original H-DETR. Have you tried both using isolated point queries and instance queries between one2one group and one2many group? BTW, Did the vanilla self-attention in the following ablation use the same setting as other self-attention variants?