Human-in-the-loop doesnt work for 3D image in Cellpose-SAM

[ynpan910]

Hi,

I am using Cellpose 4 GUI 3D mode on a 3D image. However, it does not allow me to manually draw circles after getting the automate results, or specifically, I can manually draw circles but the algorithm does not take it and it is not reflected as a cell mask, unlike on a 2D image.

On CLI, it shows nothing after I do manual annotation:

GUI_INFO: 297 masks found
GUI_INFO: creating cellcolors and drawing masks

Is there anything I should do to make it work? Thanks!

[mrariden]

Hi @ynpan910 we don't have 3D-native HITL or annotation fully working at the moment. To do HITL with 3D data you'll have to slice out your 3D data into planes using the gui/make_train.py script and then train on these using regular 2D annotations.

This is something we're working on, so check back in the future.

[ynpan910]

Hi @ynpan910 we don't have 3D-native HITL or annotation fully working at the moment. To do HITL with 3D data you'll have to slice out your 3D data into planes using the gui/make_train.py script and then train on these using regular 2D annotations.

This is something we're working on, so check back in the future.

Thanks for replying! Looking forward to the new functionalities.

[bbrence]

Hey @mrariden,

does training in 3D work this way, or should I do the slicing into 2D as well? Somehow I manage to make the finetuned model perform worse than the generic CPSAM...

# get files
output = io.load_train_test_data(train_dir, test_dir, image_filter=img_ext, mask_filter=masks_ext) # train_data and train_labels are lists of 3D volumes (same for test)
train_data, train_labels, _, test_data, test_labels, _ = output

new_model_path, train_losses, test_losses = train.train_seg(model.net,
                                                            train_data=train_data,
                                                            train_labels=train_labels,
                                                            test_data=test_data,
                                                            test_labels=test_labels,
                                                            batch_size=batch_size,
                                                            n_epochs=n_epochs,
                                                            learning_rate=learning_rate,
                                                            weight_decay=weight_decay,
                                                            nimg_per_epoch=max(2, len(train_data)), # can change this
                                                            model_name=model_name,
                                                            min_train_masks=0)

Otherwise, great tool, thank you tons!