google-cloud-node icon indicating copy to clipboard operation
google-cloud-node copied to clipboard

@google-ai/generativelanguage response with empty candidates.

Open BruceWind opened this issue 2 years ago • 1 comments

Environment details

  • which product (packages/*): @google-ai/generativelanguage
  • OS: linux
  • Node.js version: 18.18
  • npm version: 9.8.1
  • google-cloud-node version: 1.1.0

Steps to reproduce

  1. js code:
import { TextServiceClient } from '@google-ai/generativelanguage';
import { GoogleAuth } from 'google-auth-library';

const MODEL_NAME = 'models/text-bison-001';
const API_KEY = process.env.PALM_API_KEY;

const client = new TextServiceClient({
  authClient: new GoogleAuth().fromAPIKey(API_KEY)
});
...
...

  const result: any = await client.generateText({
    // required, which model to use to generate the result
    model: MODEL_NAME,
    // optional, 0.0 always uses the highest-probability result
    temperature: 0.5,
    // optional, how many candidate results to generate
    candidateCount: 1,
    // optional, number of most probable tokens to consider for generation
    topK: 40,
    // optional, for nucleus sampling decoding strategy
    topP: 0.95,
    // optional, maximum number of output tokens to generate
    maxOutputTokens: 1024,
    // optional, sequences at which to stop model generation
    stopSequences: stopSequences,
    // optional, safety settings
    safetySettings: [
      { category: 'HARM_CATEGORY_DEROGATORY', threshold: 1 },
      { category: 'HARM_CATEGORY_TOXICITY', threshold: 1 },
      { category: 'HARM_CATEGORY_VIOLENCE', threshold: 2 },
      { category: 'HARM_CATEGORY_SEXUAL', threshold: 2 },
      { category: 'HARM_CATEGORY_MEDICAL', threshold: 2 },
      { category: 'HARM_CATEGORY_DANGEROUS', threshold: 2 }
    ],
    prompt: {
      text: promptStr
    }
  });

  console.log(`result: ${JSON.stringify(result, null, 2)}`);

The log was:

result: [
  {
    "candidates": [],
    "filters": [
      {
        "reason": "SAFETY"
      }
    ],
    "safetyFeedback": [
      {
        "rating": {
          "category": "HARM_CATEGORY_DEROGATORY",
          "probability": "HIGH"
        },
        "setting": {
          "category": "HARM_CATEGORY_DEROGATORY",
          "threshold": "BLOCK_LOW_AND_ABOVE"
        }
      },
      {
        "rating": {
          "category": "HARM_CATEGORY_TOXICITY",
          "probability": "HIGH"
        },
        "setting": {
          "category": "HARM_CATEGORY_TOXICITY",
          "threshold": "BLOCK_LOW_AND_ABOVE"
        }
      },
      {
        "rating": {
          "category": "HARM_CATEGORY_VIOLENCE",
          "probability": "MEDIUM"
        },
        "setting": {
          "category": "HARM_CATEGORY_VIOLENCE",
          "threshold": "BLOCK_MEDIUM_AND_ABOVE"
        }
      }
    ]
  },
  null,
  null
]

As you can see , candidates in there was empty array.

Otherwise, If I call generativelanguge in curl, it show me non-empty and different structure in JSON:

{
  "candidates": [
    {
      "output": "**Foods**\n\n1. The grocer sells a variety of fresh fruits and vegetables.\n2. The grocer has a wide selection of meats and cheeses.\n\n**Groceries**\n\n1. The grocer sells a variety of canned goods and packaged foods.\n2. The grocer has a wide selection of cleaning supplies and toiletries.",
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_DEROGATORY",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_TOXICITY",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_VIOLENCE",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_SEXUAL",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_MEDICAL",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS",
          "probability": "LOW"
        }
      ]
    }
  ]
}

``` 

BruceWind avatar Oct 31 '23 10:10 BruceWind

And, I have tried use v1beta2 and v1beta3, it does not work. However, it had been working well 8 hours ago.

BruceWind avatar Oct 31 '23 10:10 BruceWind