dstack [Examples] Add TensorRT-LLM example (end-to-end)

Oct 01 '24 08:10 peterschmidt85

@peterschmidt85 : May I take this up?

Oct 02 '24 07:10 bikash119

@peterschmidt85 : May I take this up?

Only if you know how. It's not an easy one. TensorRT-LLM is one of the most complicated stacks. The example should show it end-to-end: how to build a model, and serve it.

Oct 02 '24 07:10 peterschmidt85

Thank you @peterschmidt85 for giving me the heads up. Will give it a try and keep you posted on how it goes. May I request you to point to any example on dstack which has similar set of task like

build the model
serve it

Oct 02 '24 11:10 bikash119

Thank you @peterschmidt85 for giving me the heads up. Will give it a try and keep you posted on how it goes. May I request you to point to any example on dstack which has similar set of task like

build the model

serve it

I would invite you to explore more what dstack is, how it works, and of course what TensorRT-LLM is and how it works.

Oct 02 '24 11:10 peterschmidt85

Thank you @peterschmidt85.

Oct 02 '24 11:10 bikash119

This issue is stale because it has been open for 30 days with no activity.

Nov 02 '24 01:11 github-actions[bot]

This issue is stale because it has been open for 30 days with no activity.

Dec 04 '24 02:12 github-actions[bot]

This issue was closed because it has been inactive for 14 days since being marked as stale. Please reopen the issue if it is still relevant.

Dec 19 '24 02:12 github-actions[bot]