PENG WENJIE

Does the model in this framework have to be trained to have conversations?

1

I am new to using LLM, and is it possible to just download the open source LM and have multiple conversations without fine-tuning?

How do models do batch inferring when using the transformer method?

6

I am a noob. Here is my code, how can I modify it to do batch inferring? --- def load_model(): model_id = 'llama3/Meta-Llama-3-70B-Instruct' pipeline = transformers.pipeline( "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16},...