BentoML icon indicating copy to clipboard operation
BentoML copied to clipboard

fix(sdk/cli): Correct HuggingFace model copy

Open b8zhong opened this issue 8 months ago • 4 comments

What does this PR address?

2 things, mainly:

  1. models/huggingface.py:

    • Corrects a bug in HuggingFaceModel.resolve where the incorrect source directory (grandparent of the snapshot) was being copied when a base_path was provided. I believe this leads to large/incorrect copies. The fix ensures only the specific model snapshot is copied to a correctly named subdirectory within the base_path
  2. service/factory.py:

    • Fixes an AttributeError in the deprecated runner_service function by initializing config["workers"] as a list ([]) instead of a dictionary ({}) 👍
  3. cli/deployment.py:

    • Removes a duplicate @click.option definition for --config-file (it's right above the deleted one 👍 )

Before submitting:

b8zhong avatar May 14 '25 01:05 b8zhong

PR Change Summary

This pull request addresses several issues related to the HuggingFace model handling and code improvements in the CLI.

  • Corrected a bug in HuggingFace model copying to ensure proper directory structure.
  • Fixed an AttributeError in the runner_service function by initializing workers correctly.
  • Removed a duplicate option definition in the CLI for better clarity.

Modified Files

  • docs/source/build-with-bentoml/services.rst
  • docs/source/examples/mlflow.rst
  • docs/source/scale-with-bentocloud/deployment/index.rst

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

Maybe also cc @aarnphm 👍 thx in advance

b8zhong avatar May 14 '25 01:05 b8zhong

Let's separate this out into 3 PRs.

The docs PR, the CLI PR, and the HF copy PR.

Can you provide the case when the base_path issue that you run into? A reproducible would be helpful here.

aarnphm avatar May 14 '25 01:05 aarnphm

Sure no problem, I made a new PR for the docs

I'll give the example in a sec

b8zhong avatar May 14 '25 03:05 b8zhong