Malformed JSON response while result from LLM call looks good

[qbloq]

Hey there! Thanks for this library.

I'm running an agent with debug: true. The first part of the output I provide here is the tail of the debug output which confirms that the LLM could predict the Agent's outputs right (Intent and Scheduling Action); however the transformation to JSON is having issues.

AGENT CALL:

    const intentExt = await customerIntentExtraction.forward(ai, {
      context, message
    }, { stream: false, debug: true })

    console.log(intentExt)

OUTPUT:

Assistant:
Intent: scheduling
Scheduling Action: inquiry

{ intent: "scheduling\nScheduling Action: inquiry" }

I'm running latest: 11.0.33

[dosco]

looking into it

[qbloq]

Not seeing more of this issue in my workflows, thanks!

I'm seeing this other incorrect parsing issue on arrays, but not being a blocker.

Assistant:
...
Dog Colors: ["Gris", "Blanco", "Negro"]
...

{
  ...
  dogColors: [ "Gris", "Blanco", "Negro", "Blanco", "Negro", "Negro" ],
  ...
}

[dosco]

Can you create a new issue for this also what model are you seeing this with seems like the model is trying to create a json object instead of just a json array.

[qbloq]

Thanks, will create a new ticket for that one.

As to this very issue, it's appearing again:

Assistant:
Intent: dog_info
Dog Name: luki
Dog Age: 2 meses
Dog Breed: golden

{
  intent: "dog_info",
  dogName: "luki\nDog Age: 2 meses",
  dogBreed: "golden"
}

It breaks when there're more than 3 outputs

[dosco]

thanks will take a second look

[dosco]

I'm unable to recreate. Here's the example I created to test this issue:

> node --env-file=.env --import=tsx src/examples/streaming-test.ts

Testing streamingForward with dog info fields...

Streaming delta: { version: 0, delta: { intent: 'Telling a story' } }
Streaming delta: { version: 0, delta: { dogName: 'Luki' } }
Streaming delta: { version: 0, delta: { dogAge: '2' } }
Streaming delta: { version: 0, delta: { dogAge: ' months' } }
Streaming delta: { version: 0, delta: { dogBreed: 'Golden Retriever' } }
Result: {
  intent: 'Telling a story',
  dogName: 'Luki',
  dogAge: '2 months',
  dogBreed: 'Golden Retriever'
}

import { AxAIGoogleGeminiModel, AxGen } from '@ax-llm/ax';
import { AxAI } from '@ax-llm/ax';

// Define the signature with the specific field names
const signature = 'story:string -> intent:string, dogName:string, dogAge:string, dogBreed:string';

// Create the generator
const gen = new AxGen<{ story: string }, { 
  intent: string;
  dogName: string;
  dogAge: string;
  dogBreed: string;
}>(signature);

// Create AI instance
const ai = new AxAI({
  name: 'google-gemini',
  apiKey: process.env.GOOGLE_APIKEY as string,
  config: { model: AxAIGoogleGeminiModel.Gemini15Flash8B },
});

// Test input
const input = {
    story: 'Once upon a time, there was a dog named Luki. Luki was a golden retriever. Luki was 2 months old.',
};

// Run the test with streamingForward
console.log('Testing streamingForward with dog info fields...\n');

const generator = gen.streamingForward(ai, input);

try {
  for await (const res of generator) {
    console.log('Streaming delta:', res);
  }

} catch (error) {
  console.error('Error during streaming:', error);
} 

try {
  const result = await gen.forward(ai, input);
  console.log('Result:', result);
} catch (error) {
  console.error('Error during forward:', error);
}

[dosco]

dogColors: [ "Gris", "Blanco", "Negro", "Blanco", "Negro", "Negro" ],

Fix committed, pushing out a new release