lmdeploy
lmdeploy copied to clipboard
[Feature] Hope the pipeline can automatically destroy resources
Motivation
I hope the pipeline can automatically manage resource destruction. Yes, I have taken on this task from the community. However, to my regret, after a period of searching and attempting, I am still unsure where to begin. I am hoping for some guidance from the community, at least to point me in the right direction of effort.
Supports pipeline resource destruction so that the following test cases can pass
models = [
'the/path/of/internlm2/model',
'the/path/of/llama3/model',
'the/path/of/qwen2/model'
]
prompts = ['hi, shanghai is', 'pls intro yourself']
for model in models:
pipe = pipeline(model)
result = pipe(prompts)
print(result)
Related resources
No response
Additional context
I'm hoping to implement this feature and create a PR, but I'm in dire need of some necessary guidance right now