Philipp Schmid

Results 136 comments of Philipp Schmid

Which versions are you using?

and transformers/datasets?

Does using session.call_tool works, without any LLM inteaction?

Hey @sck, what GPU are you using and how big is your dataset? can you try decreasing the batch_size more, like to 8 or something

Hello Streaming is not yet support on SageMaker, easyllm is not using any layer in between. I hope they ll add support soon then i ll add it as well.

Can you check out this example for sagemaker? and see if it works? https://philschmid.github.io/easyllm/examples/sagemaker-chat-completion-api/#1-import-the-easyllm-library