transformers
transformers copied to clipboard
Add DETA
What does this PR do?
This PR adds DETA. DETA is a slight change to Deformable DETR by using traditional IoU-based assignment as opposed to the Hungarian matching used in the original DETR, and incorporating NMS (non-maximum suppression) in the postprocessing.
Note: this model has a torchvision dependency for NMS.
To do:
- [ ] transfer checkpoints
cc @alaradirik this PR is in a ready state, except for 2 things:
- [x] whether or not we leverage torchvision's
batched_nms=> the CI is currently failing because this library is not installed. Will also ask for @sgugger and @LysandreJik's opinion here - [ ] the
post_process_object_detectionmethod might require an in-depth look
There is no problem with the model requiring torchvision to be installed. We have many models with specific dependencies, some of which you ported yourself ;-).
Just protect the import between if is_torchvision_available() and have a the first line in the init of the models be a require_backends(["torchvision"]).
The documentation is not available anymore as the PR was closed or merged.
@sgugger I've addressed all comments, except for adding support for the custom kernel.
Could we perhaps add support for the custom kernel for the 3 models (Mask2Former, OneFormer and DETA) in a separate PR?
In this case, remove the code trying to load the custom kernels in the modeling file and we can add it back in the PR that will deal with custom kernels.
@sgugger ok, feel free to approve :)
Failing test is unrelated/flaky, merging.