[FR] Allow custom headers in deployment_client creation

[akshaya-a]

Willingness to contribute

Yes. I can contribute this feature independently.

Proposal Summary

the easiest/best way to invoke the deployment server is via get_deploy_client(). This returns a client that internally manages headers without an easy extension point.

Consider a signature like

get_deploy_client(headers={...})

Motivation

What is the use case for this feature?

passing custom headers to the LLM invocation. In my usecase, it is to just pass a header to the deployment server, but this feature might be useful to pass to the downstream provider too through the proxy

Why is this use case valuable to support for MLflow users in general?

Why is this use case valuable to support for your project(s) or organization?

Why is it currently difficult to achieve this use case?

you have to mokeypatch in your headers like this: https://github.com/akshaya-a/mindctrl/blob/main/python/src/mindctrl/openai_deployment/init.py#L683

or use a custom client, defeating this utility

Details

No response

What component(s) does this bug affect?

• [ ] area/artifacts: Artifact stores and artifact logging
• [ ] area/build: Build and test infrastructure for MLflow
• [X] area/deployments: MLflow Deployments client APIs, server, and third-party Deployments integrations
• [ ] area/docs: MLflow documentation pages
• [ ] area/examples: Example code
• [ ] area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
• [ ] area/models: MLmodel format, model serialization/deserialization, flavors
• [ ] area/recipes: Recipes, Recipe APIs, Recipe configs, Recipe Templates
• [ ] area/projects: MLproject format, project running backends
• [ ] area/scoring: MLflow Model server, model deployment tools, Spark UDFs
• [ ] area/server-infra: MLflow Tracking server backend
• [ ] area/tracking: Tracking Service, tracking client APIs, autologging

What interface(s) does this bug affect?

• [ ] area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
• [ ] area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
• [ ] area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
• [ ] area/windows: Windows support

What language(s) does this bug affect?

• [ ] language/r: R APIs and clients
• [ ] language/java: Java APIs and clients
• [ ] language/new: Proposals for new client languages

What integration(s) does this bug affect?

• [ ] integrations/azure: Azure and Azure ML integrations
• [ ] integrations/sagemaker: SageMaker integrations
• [ ] integrations/databricks: Databricks integrations

[daniellok-db]

Sounds good, let us know when the PR is ready!

[github-actions[bot]]

@mlflow/mlflow-team Please assign a maintainer and start triaging this issue.