label-studio-ml-backend Model class instance loaded multiple times

Label-studio: 1.7.3 Label-studio-ml: 1.0.8rc3

command I use: label-studio-ml start yolov7 --kwargs hostname=<host_name:port> access_token=<API_KEY> weights_path=/workspace/LS_ML_Backend/yolov7/runs/train/yolov7-custom-15/weights/best.pt device=cuda:0 img_dir=/workspace/LS_ML_Backend/Data/Images I trained a model --> then selected that model in Label-studio UI for predictions.

Query:

Why is model being loaded every time for each prediction? Is it expected behavior? If not, could you please tell me what needs to be done. I see an issue related to this: #192

Tried & failed:

Turning off webhook

Label-studio-ml Logs:

[2023-04-26 15:16:54,462] [WARNING] [werkzeug::_log::225]  * Debugger is active!
[2023-04-26 15:16:54,479] [INFO] [werkzeug::_log::225]  * Debugger PIN: 294-335-466
[2023-04-26 15:17:04,791] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:04] "GET /health HTTP/1.1" 200 -
[2023-04-26 15:17:04,809] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
[2023-04-26 15:17:09,327] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:09] "GET /health HTTP/1.1" 200 -
[2023-04-26 15:17:09,332] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:09] "GET /health HTTP/1.1" 200 -
[2023-04-26 15:17:09,351] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
Fusing layers...
Fusing layers...
IDetect.fuse
[2023-04-26 15:17:11,828] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:17:11,829] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:11] "POST /setup HTTP/1.1" 200 -
IDetect.fuse
[2023-04-26 15:17:12,118] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:17:12,119] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:12] "POST /setup HTTP/1.1" 200 -
[2023-04-26 15:17:12,170] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:12] "GET /health HTTP/1.1" 200 -
[2023-04-26 15:17:12,178] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
Fusing layers...
IDetect.fuse
[2023-04-26 15:17:13,513] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:17:13,514] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:13] "POST /setup HTTP/1.1" 200 -
[2023-04-26 15:17:13,852] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:13] "GET /health HTTP/1.1" 200 -
[2023-04-26 15:17:13,860] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
Fusing layers...
IDetect.fuse
[2023-04-26 15:17:15,046] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:17:15,047] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:15] "POST /setup HTTP/1.1" 200 -
[2023-04-26 15:17:15,387] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:15] "GET /health HTTP/1.1" 200 -
[2023-04-26 15:17:15,395] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
Fusing layers...
IDetect.fuse
[2023-04-26 15:17:16,714] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:17:16,715] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:16] "POST /setup HTTP/1.1" 200 -
[2023-04-26 15:17:24,365] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:24] "GET /health HTTP/1.1" 200 -
[2023-04-26 15:17:24,380] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
Fusing layers...
IDetect.fuse
[2023-04-26 15:17:25,553] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:17:25,554] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:25] "POST /setup HTTP/1.1" 200 -
[2023-04-26 15:17:32,392] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:32] "GET /health HTTP/1.1" 200 -
[2023-04-26 15:17:32,399] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
Fusing layers...
IDetect.fuse
[2023-04-26 15:17:33,782] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:17:33,783] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:33] "POST /setup HTTP/1.1" 200 -
[2023-04-26 15:17:33,796] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
Fusing layers...
IDetect.fuse
[2023-04-26 15:17:34,859] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:17:35,449] [DEBUG] [yolov7_ml_backend::_infer::855] (567.0ms) Inference, (2.8ms) NMS
[2023-04-26 15:17:35,452] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:17:35] "POST /predict HTTP/1.1" 200 -
[2023-04-26 15:19:15,803] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:19:15] "GET /health HTTP/1.1" 200 -
[2023-04-26 15:19:15,815] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
Fusing layers...
IDetect.fuse
[2023-04-26 15:19:17,032] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:19:17,032] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:19:17] "POST /setup HTTP/1.1" 200 -
[2023-04-26 15:19:17,048] [DEBUG] [yolov7_ml_backend::__init__::146] Loaded model from /workspace/LS_ML_Backend/yolov7/runs/train/Model-1/weights/best.pt
Fusing layers...
IDetect.fuse
[2023-04-26 15:19:18,344] [DEBUG] [yolov7_ml_backend::__init__::153] TextYOLODetection - model loaded!
[2023-04-26 15:19:18,475] [DEBUG] [yolov7_ml_backend::_infer::855] (115.0ms) Inference, (1.8ms) NMS
[2023-04-26 15:19:18,477] [INFO] [werkzeug::_log::225] 172.17.0.2 - - [26/Apr/2023 15:19:18] "POST /predict HTTP/1.1" 200 -

Apr 26 '23 10:04 KD1994

Facing same issue. @makseq Please take a look.

Apr 27 '23 05:04 AkashDataScience

I am also having this problem.

Apr 29 '23 07:04 TrueWodzu

I am having this problem as well

May 17 '23 14:05 rkabealo

I currently use singleton to avoid this problem.

def singleton(cls):
    _instance = {}

    def inner():
        if cls not in _instance:
            _instance[cls] = cls()
        return _instance[cls]
    return inner

@singleton
class DetectorCls(object):
    def __init__(self):
        print('Detector init')

class NewModel(LabelStudioMLBase):
    def __init__(self, project_id, **kwargs):
        super(NewModel, self).__init__(project_id, **kwargs)
        self.detector = DetectorCls()

Nov 08 '23 08:11 kensma