Support for post_process_semantic_segmentation added

[shaddu]

Description

post_process_segmentation is getting depreciated in transformers version 5. Code changes added are to support post_process_semantic_segmentation

Type of change

Please delete options that are not relevant.

• [ ] New feature (non-breaking change which adds functionality)

How has this change been tested, please provide a testcase or example of how you tested the change?

Detection test Colab

Any specific deployment considerations

No

[CLAassistant]

All committers have signed the CLA.

[SkalskiP]

Hi @shaddu 👋🏻 Thank you for submitting the PR. I have a few questions regarding the scope of the changes that have been and will need to be made in from_transformers to fully migrate us. In https://github.com/roboflow/supervision/issues/1113 I see that we need to migrate post_process, post_process_panoptic, post_process_segmentation and post_process_instance. Do I understand correctly that this PR only migrates post_process_segmentation for now, and the remaining outputs still need to be migrated?

[shaddu]

Hi @SkalskiP , Thank you for reviewing the PR and providing your comments. The scope was to migrate post_process, post_process_panoptic, post_process_segmentation and post_process_instance but I have recently started working on computer vision, so I wanted to make sure before making all the changes that I am in the right direction. If this code is fine then I can migrate the rest of the method in this PR only or another as you prefer.

[SkalskiP]

@shaddu I like what you have done so far! Both code quality and functionality look good to me. From my perspective, the most important thing is that from_transformers supports both v4 and v5.

I think you can continue the migration in this PR.

[shaddu]

Hi @SkalskiP ,

I have finished adding support for the remaining function. Could you please review it and offer feedback?

Thank you

[LinasKo]

Hi @shaddu

Thank you for your contribution! I'll check this today; hopefully we can merge soon.

[SkalskiP]

Hi @shaddu 👋🏻 Awesome work here. I left some comments regarding the current version of the code, but here are some high-level overviews ones:

• Could you update your Colab? The current version tests only a limited subset of supported capabilities. It would be awesome to test all functions mentioned in https://github.com/roboflow/supervision/issues/1113 in a single Colab.
• In my opinion, having all that logic in a single function is hard to follow. I would create a new file, supervision/detection/transformers.py, and inside define process_transformers_v5_result, process_transformers_v4_detection_result, process_transformers_v4_segmentation_result, etc, and just call those inside from_transformers in supervision/detection/core.py. This will:
  • Make supervision/detection/core.py shorter. It is already too long.
  • Make transformers' logic organized and easier to maintain.

In general this looks really good already. Thanks a lot for all the help! 🙏🏻

[LinasKo]

Hi @shaddu,

Here's a Colab that may help as a starting point. I've got examples of every segmentation type, except for instance segmentation.

It would be great to also test with a few other models - not just a panoptic segmentation one.

Lastly, I'm curious: does panoptic segmentation add anything to the detections.data field?

[shaddu]

Hey @SkalskiP,

Thank you for the detailed feedback. The suggestion of a separate file is a good idea. I'll work on this approach.

Hey @LinasKo,

Thank you for sharing the new collab. I learned something new today on how to use your repos in collab notebook 👍

[shaddu]

Hi @SkalskiP / @LinasKo ,

I have made changes as per the feedback, can you please review the PR again?

@LinasKo post_process_panoptic_segmentation adds class names in detection.data.

I will be adding more models and test cases in this colab for review.

[SkalskiP]

Hi @shaddu 👋🏻 ! Awesome work! I added small changes - changed names of functions and improved docstrings. We are ready to merge! Thanks a lot for the contribution! 🙏🏻 Supporting transformers is one of our goals; your contribution is a big step in that direction.

[shaddu]

@SkalskiP Thank you for merging the PR. It was a great learning experience. I'll look for more issues to work on in Supervision and if you come across any related issues, I'd be happy to help with them.

[SkalskiP]

@shaddu, working with you was an awesome experience! 🔥