ghostplant comments

Repositories
Issues
Comments

Results 272 comments of


                                            ghostplant

[Question] How can I set the max tokens with tutelgroup/deepseek-671b

But it is a good feature request and we'll handle those settings through REST API in the future.

[Question] How can I set the max tokens with tutelgroup/deepseek-671b

The new version of `tutelgroup/deepseek-671b:mi300x8-chat-20250723` allows setting `max_tokens` in REST: ```sh curl -X POST http://0.0.0.0:8000/chat -d '{"messages": [{"role": "user", "content": "Hello."}], "max_tokens": 256 }' ```