keras-cv icon indicating copy to clipboard operation
keras-cv copied to clipboard

Published 20 hours ago • verified publisher icon keras-team

Support for augmention in segmentation task

Open old-school-kid opened this issue 2 years ago • 13 comments

Short Description Sort of duplicate of #16117

RandomRotation has utility for augmenting bounding boxes which is again missing in random shear layer. Also there is no support for augmenting target in case of a segmentation task.

Other Information possible solution: augment_label function can be used to check if the label is an image and carry out the same augmentation as on image.

old-school-kid avatar Jun 27 '22 08:06 old-school-kid

Hey @LukeWood, I and @soumik12345 would like to take this up. Since there is no contribution-welcome label added to this issue, would first love to get a green light before proceeding.

ayulockin avatar Jul 14 '22 08:07 ayulockin

Hey @ayulockin thanks for checking in on this.

If you can provide an in depth overview of the different segmentation map formats we can start with utilities to convert between formats.

Does that sound like a plan?

LukeWood avatar Jul 14 '22 17:07 LukeWood

That sounds like a plan. Should we have a different issue to capture it? I can create one and self assign + assign Soumik.

ayulockin avatar Jul 14 '22 17:07 ayulockin

No thanks, I can just assign to you for now. Lets keep it here.

LukeWood avatar Jul 14 '22 18:07 LukeWood

Soumik must comment to be assigned!

LukeWood avatar Jul 14 '22 18:07 LukeWood

@LukeWood

If you can provide an in depth overview of the different segmentation map formats we can start with utilities to convert between formats.

There may be three most common formats:

  • png as image format,
  • polygon and
  • rle https://github.com/keras-team/keras-cv/issues/327.

cc. @ayulockin @old-school-kid @soumik12345

innat avatar Jul 15 '22 01:07 innat

@LukeWood

If you can provide an in depth overview of the different segmentation map formats we can start with utilities to convert between formats.

There may be three most common formats:

cc. @ayulockin @old-school-kid @soumik12345

Cc @sayakpaul @tanzhenyu @innat

LukeWood avatar Jul 15 '22 02:07 LukeWood

I agree with @innat.

The proposal that @innat made in https://github.com/keras-team/keras-cv/issues/327 is really solid. I see two ways:

  • We first create the RLE utilities. Then work on supporting different Segmentation formats.
  • We first develop utilities to support the first two formats - images and polygons as they are relatively easier.

This way we should be able to better isolate the components. But keen to know what others think.

sayakpaul avatar Jul 15 '22 02:07 sayakpaul

I agree with the suggestion @sayakpaul made. Supporting rle2mask2d and mask2d2rle should be an excellent place to begin.

ayulockin avatar Jul 15 '22 04:07 ayulockin

Note we'd prefer rle_to_mask2d() and mask2d_to_rle() to the ones you mentioned. Minor nit but just wanted to mention.

sayakpaul avatar Jul 15 '22 04:07 sayakpaul

So, do we consider these two formats to both be segmentation map formats, or are they distinct label types?

LukeWood avatar Jul 15 '22 05:07 LukeWood

@LukeWood could you elaborate more? Couldn't really get your question.

sayakpaul avatar Jul 15 '22 05:07 sayakpaul

Hey @LukeWood, I and @soumik12345 would like to take this up. Since there is no contribution-welcome label added to this issue, would first love to get a green light before proceeding.

Sounds good to me. I marked this as contribution-welcome. Though I'd like to point out our contribution guideline, especially as this introduces new API, it's important to have one or two rounds of API Design before making code changes.

tanzhenyu avatar Jul 15 '22 15:07 tanzhenyu

I think this is all addressed by @ianstenbit in https://github.com/keras-team/keras-cv/pull/748 and https://github.com/keras-team/keras-cv/pull/751

qlzh727 avatar Oct 21 '22 17:10 qlzh727