openai-cookbook icon indicating copy to clipboard operation
openai-cookbook copied to clipboard

flags sensitivity too low in moderation api

Open mattforgh opened this issue 2 years ago • 0 comments

Identify the file to be fixed As we can see in the documentation here the api flags true "I want to kill them." but not other bad words like above :

Describe the problem flags sensitivity too low

Describe a solution

text: fuck you  // my console log
The content is not ambiguous // my console log 
[
  {
    flagged: false,
    categories: {
      sexual: false,
      hate: false,
      violence: false,
      'self-harm': false,
      'sexual/minors': false,
      'hate/threatening': false,
      'violence/graphic': false
    },
    category_scores: {
      sexual: 0.0003065843,
      hate: 0.009011519,
      violence: 0.001343719,
      'self-harm': 0.00006550554,
      'sexual/minors': 0.0001045709,
      'hate/threatening': 0.0000513273,
      'violence/graphic': 0.00011286236
    }
  }

] maybe I'm not getting something, let me know however this is the way how i call the API :

   const promtpData = text;
    const moderationData = {
      model: 'text-moderation-stable',
      input: promtpData
    };


    const moderationHeaders = {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`
    };

    // Send moderation request
    const moderationResponse = await axios.post(
      'https://api.openai.com/v1/moderations',
      moderationData,
      { headers: moderationHeaders }
    );

mattforgh avatar Jun 06 '23 13:06 mattforgh