segment-anything
segment-anything copied to clipboard
facebookresearch
Tutorial: fine-tune SAM on custom dataset (demo notebooks)
Hi,
SAM was recently added to Hugging Face Transformers 🤗 , and I made some tutorial notebooks to showcase how to fine-tune the model on a different domain (medical images) as well as inference:
https://github.com/NielsRogge/Transformers-Tutorials/tree/master/SAM
Thanks for sharing. Would like to ask, how long does your fine-tune training process take?
load a small dataset of 130 (image, ground truth mask) pairs.
And is the result good?
(I am thinking to fine-tune for detecting objects from satellite/drone images)
how long does your fine-tune training process take?
For me it only took a couple of minutes as my dataset only contains 130 images.
The result is good yes, see also the MedSAM project + paper which fine-tunes SAM on the medical domain, thereby improving performance.
@NielsRogge Hi Niels. I was wondering if there is a way to download your dataset (130 images with masks) on local. load_dataset("nielsr/breast-cancer", split="train") this line loading a dataset, but I would like to download to see the images and masks. Can you please tell me where I can download the ZIP file of breast-cancer dataset. Thanks a lot in advance!
By default, the Datasets library (like Transformers) stores everything in your local cache.
You can always store it using the save_disk method: https://huggingface.co/docs/datasets/process#save.
@NielsRogge thanks for your quick reply. I used save_disk and in my local 3 files are saved
data-00000-of-00001.arrow dataset_info.json state.json
Can you please let me know how I can extract images and masks files using these 3 files. Thanks in advance!
@NielsRogge This is truly an amazing project! I have a few questions I would like to ask:
- Can I still use the approach provided in this project for fine-tuning if the images in my dataset have different sizes?
- Are there any restrictions on the sizes of the images in the dataset?
i had following error while preparing custom dataset for color image .
ValueError: zero-size array to reduction operation minimum which has no identity
