gptel icon indicating copy to clipboard operation
gptel copied to clipboard

User input attributed to "assistant" role triggering "Start of structure or map found where not expected"

Open leezu opened this issue 9 months ago • 2 comments

Please update gptel first -- errors are often fixed by the time they're reported.

  • [x] I have updated gptel to the latest commit and tested that the issue still exists

Bug Description

gptel currently mixes up :role "user" and :role "assistant" content when editing a org buffer containing both user and assistant content.

For example, a query with only user content works fine:

(:messages
 [(:role "user" :content
         [(:text "#+TITLE: Test
OPENED: [2025-06-30 Mon 11:08]

Test user input")])]
 :inferenceConfig
 (:maxTokens 500)
 :system
 [(:text "You are a large language model living in Emacs and a helpful assistant. Respond concisely.")]
 :temperature 1.0)

This yields the following document

Test user input

I can see you have an Org-mode document with:

- Title: "Test"
- An OPENED timestamp from June 30, 2025 at 11:08 AM
- Content: "Test user input"

How can I help you with this document? Would you like me to help format it, add content, or assist with something specific about Org-mode?

A follow-up query containing the text "Modified input inside of the answer" added into the middle of the model response, triggers "Start of structure or map found where not expected" in the gptel-log buffer and no response from the model. Inspecting the request, we see that the modified input is assigned incorrectly to the "assistant" role and a apparently invalid "user" content with :text nil is used:

(:messages
 [(:role "user" :content
         [(:text "#+TITLE: Test
OPENED: [2025-06-30 Mon 11:08]

Test user input")])
  (:role "assistant" :content
         [(:text "I can see you've shared an Org mode document with:

- Title: \"Test\"
- Opened timestamp: June 30, 2025, Monday at 11:08
- Content: \"Test user input\"")])
  (:role "user" :content
         [(:text nil)])
  (:role "assistant" :content
         [(:text "Modified input inside of the answer

Is there something specific you'd like me to help you with regarding this document or Org mode in general?")])
  (:role "user" :content
         [(:text nil)])]
 :inferenceConfig
 (:maxTokens 500)
 :system
 [(:text "You are a large language model living in Emacs and a helpful assistant. Respond concisely.")]
 :temperature 1.0)

For reference, the error is triggered with the following document:

Test user input

I can see you have an Org-mode document with:

- Title: "Test"
- An OPENED timestamp from June 30, 2025 at 11:08 AM
- Content: "Test user input"

Modified input inside of the answer

How can I help you with this document? Would you like me to help format it, add content, or assist with something specific about Org-mode?

Backend

None

Steps to Reproduce

  1. Trigger gptel response in org buffer
  2. Modify the response inline
  3. Trigger a new completion and observe the error

Additional Context

Emacs 29.3 Operating system: Ubuntu 24.04

Backtrace


Log Information


leezu avatar Jun 30 '25 15:06 leezu

Which backend and model are you using here?

karthink avatar Jul 04 '25 01:07 karthink

@karthink this is with the bedrock backend and claude-sonnet-4-20250514 model and using https://github.com/doomemacs/doomemacs/.

leezu avatar Jul 11 '25 00:07 leezu