pymlup icon indicating copy to clipboard operation
pymlup copied to clipboard

[Feature] Add support hugging face to storage type

Open nxexox opened this issue 2 years ago • 0 comments

Why

Use models from hugging face in config.

Describe the solution you'd like

Add storage class for hugging face and load models.

# mlup/ml/storage/hugging_face.py
from dataclasses import dataclass
from typing import List

from mlup.ml.storage.base import BaseStorage
from mlup.constants import LoadedFile

@dataclass
class HuggingFaceStorage(BaseStorage):
    repo_name: str
    files: List[str]
    need_load_file: bool = False

    def load_bytes_single_file(self, path_to_file: str) -> LoadedFile:
        # Load file from hugging face to local disk and memory
        return LoadedFile(path=path_to_file)

    def load(self) -> List[LoadedFile]:
        ...
...
ml:
    storage_type: mlup.ml.storage.hugging_face.HuggingFaceStorage
...

nxexox avatar Nov 07 '23 11:11 nxexox