lmdeploy icon indicating copy to clipboard operation
lmdeploy copied to clipboard

[Feature] Hope the pipeline can automatically destroy resources

Open Volta-lemon opened this issue 1 year ago • 0 comments

Motivation

I hope the pipeline can automatically manage resource destruction. Yes, I have taken on this task from the community. However, to my regret, after a period of searching and attempting, I am still unsure where to begin. I am hoping for some guidance from the community, at least to point me in the right direction of effort.

Supports pipeline resource destruction so that the following test cases can pass

models = [
    'the/path/of/internlm2/model',
    'the/path/of/llama3/model',
    'the/path/of/qwen2/model'
]

prompts = ['hi, shanghai is', 'pls intro yourself']

for model in models:
    pipe = pipeline(model)
    result = pipe(prompts)
    print(result)

Related resources

No response

Additional context

I'm hoping to implement this feature and create a PR, but I'm in dire need of some necessary guidance right now

Volta-lemon avatar Sep 23 '24 09:09 Volta-lemon