Repeated properties in vllm + xgrammar

[denadai2]

Is it normal that constrained properties could be repeated a lot?

from pydantic import BaseModel
class EpisodeGuestExtractionResponse(BaseModel):
    guests: List[str]

This is not a valid json: :/

{ "guests": ["David Farrier", "Monica Padman] }  (no guests)  { " 

  , 
  "guests" 
  , 
  "guests" 
  , 
  "guests" 
  , 
  "guests" 
  , 
  "guests" 
  , 
  "guests" 
  , 
  "guests" 

thx

[Ubospica]

Hi @denadai2 , thanks for reporting this issue! Sorry for the late response because I was occupied with some paper-related work.

Your output still seems to be valid JSON, but just its content is not expected. One possible reason for that is you may not describe the output structure in the prompt, but only rely on constraint decoding. This may not work well because LLM is not aware of the constraint. If this is the case, you can describe the structure in the prompt, say, "output a json object that contains a key 'guests' with a value that is a list of strings". That is helpful in practice.

Otherwise, could you provide your specific model and prompt? I can dig into that to see what the problem is.