Blank lines when writing code

[AndyMik90]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.8

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

Make Gpt-4o or similar LLM's write long code. It looks like it just pushes a lot of blank lines when it reaches the context window?

✔️ Expected Behavior

Either stop at the context window or finish writing the code.

❌ Actual Behavior

a lot of blank lines (see video)

https://github.com/langgenius/dify/assets/119136210/fc5bcb09-7af8-4522-8b81-a479c0b3c343

[dosubot[bot]]

To address the issue of unnecessary blank lines when using Gpt-4o or similar LLMs for writing long pieces of code, especially at the context window limit, consider adjusting the response_format parameter in your API call. Opting for a json_object format for responses might help in structuring the output more effectively, which could reduce the occurrence of blank lines. This format allows for parsing and formatting on the client side, giving you control over the output presentation, including handling blank lines.

Here's how you might adjust your API call:

response_format: json_object

Afterwards, on the client side, parse this JSON object and format it into your desired code output, managing blank lines as needed. This approach requires additional client-side handling to parse the JSON object and format it into code, but it offers more control over the output.

Regarding the context window limit, the Gpt-4o model in Dify version 0.6.8 has a token limit of 128,000. This limit is the maximum amount of information the model can consider for generating a response at any given time. The insertion of blank lines when this limit is reached is not explicitly documented, but adjusting the response format as suggested might mitigate the issue.

Details

Sources

• api/core/model_runtime/model_providers/openai/llm/gpt-4o.yaml

About Dosu This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[iamjoel]

I use the gpt-4o but i can't reproduce the issue. Can you support more detail to reproduce?