Brosoul comments

Results 10 comments of


                                            Brosoul

Should ModelAdapter displays the `Status` on the result of get command

> you mean implementing a plugin to show the status? Yes @Jeffwan

Add overall timeout support

I'm really looking forward to this feature. Could you please ping me after PR is ready?

StreamLoader supports each TP process to only read the corresponding part of the model file

Let's put the issue that about streamLoader or performance optimization into later versions, like v0.3.0

StreamLoader splits tensors more evenly

The final form should be https://github.com/aibrix/aibrix/issues/401

StreamLoader library support device_map to allocate tensor to different device

This task may be a prerequisite for https://github.com/aibrix/aibrix/issues/403

StreamLoader library support device_map to allocate tensor to different device

Let's put the issue that about streamLoader or performance optimization into later versions, like v0.3.0

Speedup downloading models by parallel downloading of multiple files

Let's put the performance optimization into later versions, like v0.3.0

StreamLoader support multi processes and multi threads

Let's put the issue that about streamLoader or performance optimization into later versions, like v0.3.0

[RFC] Model management in Runtime

## Alternatives Considered 1. Distinguish the status of model files Currently, it is possible to distinguish whether a file has been downloaded through the `metadata` file under `.cache/`. However, it...

[RFC] Model management in Runtime

It seems that Huggingface used `xxx.lock` file with `FileLock` in [filelock](https://py-filelock.readthedocs.io/en/latest/) to control the locking during the download process. cc @Jeffwan refs: [huggingface source code](https://github.com/huggingface/huggingface_hub/blob/6bfa5dd44d0472909dc778d30fbaac92e84ff96e/src/huggingface_hub/utils/_fixes.py#L95C5-L95C77)