franz101 comments

Results 63 comments of


                                            franz101

mistral-nemo:12b-instruct-2407-fp16 will return empty string using json mode while mistral-nemo:12b will return code

Sure thing: https://colab.research.google.com/drive/1wFO7-A3iRMwjcLDi_W3cSniBu1lTid6T?usp=sharing You will see only the normal model works here

mistral-nemo:12b-instruct-2407-fp16 will return empty string using json mode while mistral-nemo:12b will return code

thanks for reproducing it, basically the quantized version would output the content and the smarter model the actual function call :D

mistral-nemo:12b-instruct-2407-fp16 will return empty string using json mode while mistral-nemo:12b will return code

``` ChatCompletionMessage(content='', refusal=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_exfl6bms', function=Function(arguments='{"text":"world"}', name='type'), type='function')]) ```

mistral-nemo:12b-instruct-2407-fp16 will return empty string using json mode while mistral-nemo:12b will return code

yes hasty mistake: print(response.choices[0].message.content.strip())

mistral-nemo:12b-instruct-2407-fp16 will return empty string using json mode while mistral-nemo:12b will return code

System prompt: `You are an assistant that always performs a function call`

Does related queries work? I keep getting 429

My personal workaround for similar data is to use the wikipedia api: [see python wikipedia api for page views](https://substack.com/@franz101/p-148522892)

Implement new JSON data type.

Amazing PR! Is there benchmark to compare the two versions?

Using v2.6 model with video on Mac

how about streaming?

Add Bfloat16 support

@xzkostyan mind approving this?

feat(type): support dynamic column

How do I set `allow_experimental_dynamic_type = 1` in the test?