Finetuning SAM for customized segmentation

[Mohit0928]

I'm seeking a method to fine-tune SAM for generating custom segments. Below, you'll find two segmented images of a chair. I've employed a specific logic for segmentation (where each color represents a segment).

Segmented Image 1:
Segmented Image 2:
I went through a lot of SAM fine-tuning procedures, but all of them use binary or multiclass masks. When it comes to fine-tuning for segmentation, there is no straightforward method for defining labels for the masks. This is because it's possible that the ground truth contains 5 masks while the predicted output contains 10 masks. Consequently, it becomes challenging to compare them and optimize the process.

My segmentation approach involves the transformation of a 3D chair into a 2D representation. Each segmented part can be isolated and then unwrapped into a 2D format. I am segmenting the 3D chair in a manner that allows each segmented part to be presented as a 2D representation.

[szym1998]

Did you find the solution?