Button "start training" sent event "PROJECT_UPDATED" rather than "START_TRAINING" to ml backend

[g811201]

Describe the bug I use label-studio-ml backend to create a active learning loop. I would like to train model only when I click "Start Training" button. However, when I click the button, the ml-backend event in fit method get "PROJECT_UPDATED" rather than "START_TRAINING". I'm not sure the issue belongs to label-studio or label-studio-ml, or is there any way to achieve the goal?

To Reproduce Steps to reproduce the behavior:

1. Go to 'Project'-->'Settings'-->'Model'-->'...'
2. Click on 'Start Training'
3. ML backend log in fit method get 'PROJECT_UPDATED'

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• label-studio active on docker
• Label Studio Version [1.12.1]
• label-studio-ml 2.0.1.dev0

[makseq]

@g811201 Are you sure you are using the latest LS version?

https://github.com/HumanSignal/label-studio/blob/1.12.1/label_studio/ml/api_connector.py#L194

1. Check /version page, is it 1.12.1 ?
2. Check /feature-flags page, do you see ff_back_dev_1417_start_training_mlbackend_webhooks_250122_long enabled? If not, try to run LS: ff_back_dev_1417_start_training_mlbackend_webhooks_250122_long=1 label-studio.

[g811201]

@g811201 Are you sure you are using the latest LS version?

https://github.com/HumanSignal/label-studio/blob/1.12.1/label_studio/ml/api_connector.py#L194

1. Check /version page, is it 1.12.1 ?

2. Check /feature-flags page, do you see ff_back_dev_1417_start_training_mlbackend_webhooks_250122_long enabled? If not, try to run LS: ff_back_dev_1417_start_training_mlbackend_webhooks_250122_long=1 label-studio.

I have re-pulled the v1.12.1 image, now when I click "start training", I get "POST /webhook HTTP/1.1" 200 -, but cannot active my fit function in label-studio-ml backend.

[Massad]

@g811201 I believe you would need to add 'START_TRAINING' here: https://github.com/HumanSignal/label-studio-ml-backend/blob/master/label_studio_ml/api.py#L113

[g811201]

@Massad Thanks a lot! You're right, I should add 'START_TRAINING' in ./label_studio_ml/api.py. I added "START_TRAINING" in label_studio_ml/model.py before and it did not work.

But now there is another issue. When I active "Start model training on annotation submission" button on label studio UI and submit a new annotation, label studio will send event "START_TRAINING" first, then send "ANNOTATION_CREATED". It will trigger model.fit twice. Is there anyway can trigger model.fit separately?

[makseq]

You can trigger it manually - go to Model in the project settings, click on the three dots on the model card. You will see Start training action.

[sajarin]

Closing this issue due to inactivity, feel free to message me on Slack or post in the Discourse