Have different install options to skip installing UI packages (matplotlib & mplcursors) for a production environment

[frallain]

🚀 The feature

On a production machine that runs ML algorithms, the UI oriented packages are not necessary to make the algorithms run.

It would be nice to be able to skip these UI oriented packages installation by grouping these UI packages into a ui group in the ppyproject.ml file so that we can then perform a poetry install --without ui.

Motivation, pitch

The motivation here is to avoid UI oriented packages dependencies problem on production machines.

Alternatives

No response

Additional context

No response

[frgfm]

Hi @frallain :wave:

I agree that this is a topic, and we've discussed this several times. Here is my point of view:

• yes we should separate non-critical deps. For a maintenance perspective (to prioritize core features/issues against optional ones), they should be moved to optional extras
• however, for production, that would be extremely useful to have your feedback: what are your specific use cases in production? (I'm asking because we try to move everything to ONNX for production to avoid heavy deps and unoptimized DL operations)

[frallain]

My specific usecase in production is the following:

from doctr.models.detection import db_resnet50
from doctr.models.detection.predictor import DetectionPredictor
from doctr.models.predictor import OCRPredictor
from doctr.models.preprocessor import PreProcessor
from doctr.models.recognition import crnn_vgg16_bn
from doctr.models.recognition.predictor import RecognitionPredictor

...

def get_ocr_model():
    if MachineLearningModelType.OCR_MODEL not in MACHINE_LEARNING_MODELS:

        detection_model = db_resnet50(
            pretrained_backbone=False  # needed to not download default weights
        )

        detection_predictor = DetectionPredictor(
            PreProcessor(
                detection_model.cfg["input_shape"][:-1],
                mean=detection_model.cfg["mean"],
                std=detection_model.cfg["std"],
                batch_size=1,
            ),
            detection_model,
        )

        download_model_files("db_resnet50")

        detection_predictor.model.load_weights(
            os.path.join(settings.MODELS_DIR, "ocr_model/db_resnet50/weights")
        )

        recognition_model = crnn_vgg16_bn(
            pretrained_backbone=False  # needed to not download default weights
        )

        recognition_predictor = RecognitionPredictor(
            PreProcessor(
                recognition_model.cfg["input_shape"][:2],
                preserve_aspect_ratio=True,
                mean=recognition_model.cfg["mean"],
                std=recognition_model.cfg["std"],
                batch_size=32,
            ),
            recognition_model,
        )

        download_model_files("crnn_vgg16_bn")

        recognition_predictor.model.load_weights(
            os.path.join(settings.MODELS_DIR, "ocr_model/crnn_vgg16_bn/weights")
        )

        # For now it will download the weights for a crop orientation model that is never used
        # but this will be fixed in the next release of doctr
        MACHINE_LEARNING_MODELS[MachineLearningModelType.OCR_MODEL] = OCRPredictor(
            detection_predictor, recognition_predictor
        )

    return MACHINE_LEARNING_MODELS[MachineLearningModelType.OCR_MODEL]

ocr_model = get_ocr_model()
result = ocr_model([straight_image])
ocr_result = result.export()

[frgfm]

Alright, so from what I can see, you need:

• preprocessors (doable with PIL & numpy)
• models (doable with ONNX)
• postprocessor (doable with cv2 & numpy)
• export method (pure python)

So here the trick is about cv2 I think. A naive version would be:

• PIL + numpy + ONNX + cv2

and hopefully by having either put this in ONNX (converting to TF/PyTorch ops then exporting as ONNX), or numpy (trickier), we can have:

• PIL + numpy + ONNX

The latter is a very fast & light production environment I've been experimenting with on other projects. Let us check how we could do this & come back to you :+1:

[frgfm]

related to #1064