[FR] [Roadmap] Support custom docker images for serving

[BenWilson2]

MLflow Roadmap Item

This is an MLflow Roadmap item that has been prioritized by the MLflow maintainers. We’ve identified this feature as a highly requested addition to the MLflow package based on community feedback. We're seeking a community contribution for the implementation of this feature and will enthusiastically support the development and review of a submitted PR for this.

Contribution Note

As with other roadmap items, there may be a desire for multiple contributors to work on an issue. While we don’t discourage collaboration, we strongly encourage that a primary contributor is assigned to roadmap issues to simplify the merging process. The items on the roadmap are of a high priority. Due to the wide-spread demand of roadmap features, we encourage potential contributors to only agree to take on the work of creating a PR, making changes, and ensuring that test coverage is adequately created for the feature if they are willing and able to see the implementation through to a merged state.

Feature scope

This roadmap feature’s complexity is classified as:

• [ ] good-first-issue: This feature is limited in complexity and effort required to implement.
• [ ] simple: This feature does not require a large amount of effort to implement and / or is clear enough to not need a design discussion with maintainers.
• [X] involved: This feature will require a substantial amount of development effort but does not require an agreed-upon design from the maintainers. The feedback given during the PR phase may be involved and necessitate multiple iterations before approval. (Please bear with us as we collaborate with you to make a great contribution)
• [ ] design-recommended: This is a substantial feature that should have a design document approved prior to working on an implementation (to save your time, not ours). After agreeing to work on this feature, a maintainer will be assigned to support you throughout the development process.

Proposal Summary

As alluded to in #1963 , #5170 , #4617, and #4596, we would like to support custom 'base image' support for model serving.

At its core, this feature should:

1. Add support for providing a base image definition when building the docker image for a model artifact.
2. Generate the required python environment using the defined virtualenv as configured in the model artifact's directory.

As a reference, a PR was filed several years ago that may prove quite useful: #2189 when implementing this feature.

Note: It may be advantageous to supply a simple RFC prior to working on this to minimize the amount of change requests.

Motivation

What is the use case for this feature?

To support custom dependencies required for model serving, to customize the environment that the served model runs in (i.e., add logging support at the serving endpoint, add the ability to containerize custom libraries and file dependencies, define environment variables, etc.).

Why is this use case valuable to support for MLflow users in general?

This has been requested many times over the years. We agree that it is a very useful feature to have.

What component(s), interfaces, languages, and integrations does this feature affect?

Components

• [ ] area/artifacts: Artifact stores and artifact logging
• [ ] area/build: Build and test infrastructure for MLflow
• [ ] area/docs: MLflow documentation pages
• [ ] area/examples: Example code
• [ ] area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
• [X] area/models: MLmodel format, model serialization/deserialization, flavors
• [ ] area/projects: MLproject format, project running backends
• [ ] area/scoring: MLflow Model server, model deployment tools, Spark UDFs
• [ ] area/server-infra: MLflow Tracking server backend
• [ ] area/tracking: Tracking Service, tracking client APIs, autologging

Interfaces

• [ ] area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
• [X] area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
• [ ] area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
• [ ] area/windows: Windows support

Languages

• [ ] language/r: R APIs and clients
• [ ] language/java: Java APIs and clients
• [ ] language/new: Proposals for new client languages

Integrations

• [ ] integrations/azure: Azure and Azure ML integrations
• [ ] integrations/sagemaker: SageMaker integrations
• [ ] integrations/databricks: Databricks integrations

[BenWilson2]

For any questions, concerns, or clarification on implementing this issue, please ping @WeichenXu123

[rafaelvp-db]

@BenWilson2 @WeichenXu123 if you're looking for contributors I'm also happy to work on this one

[subramaniam02]

@BenWilson2 @dbczumar What is the scope of this feature request ? Currently ubuntu is the only OS flavor supported, do we want to support more flavors than ubuntu ? if yes, what is the plan to handle different package installer i.e, yum, apt-get, etc ?

RUN apt-get -y update
RUN apt-get install -y --no-install-recommends \
         wget \
         curl \
         nginx \
         ca-certificates \
         bzip2 \
         build-essential \
         cmake \
         openjdk-8-jdk \
         git-core \
         maven \
    && rm -rf /var/lib/apt/lists/*

[fzhcary]

Is this serving python or R models? I am interested

[dbczumar]

@anuarkaliyev23 has addressed this item by introducing an mlflow models generate-dockerfile API (#6591) that generates a complete Dockerfile for a given model or a Dockerfile for a generic serving image, if --model-uri is omitted. Users can then customize this Dockerfile to their liking before building an imge. Awesome work, @anuarkaliyev23 !