nlp.js icon indicating copy to clipboard operation
nlp.js copied to clipboard

useNoneFeature not working

Open reinoute opened this issue 3 years ago • 1 comments

According to the most basic example on the Github page, the library should return weighted scores for the intents and include a None intent. However, it only returns 1 and 0 scores and not a None intent.

Unless I'm doing something wrong (in configuration?), I believe this is a major bug.

To Reproduce I copy pasted the basic example in a runkit, which will show these scores:

  classifications:
   [ { label: 'greetings.bye', value: 1 }],
  intent: 'greetings.bye',
  score: 1,

Expected behavior According to the docs, it should show these scores:

  classifications:
   [ { label: 'greetings.bye', value: 0.698219120207268 },
     { label: 'None', value: 0.30178087979273216 },
     { label: 'greetings.hello', value: 0 } ],
  intent: 'greetings.bye',
  score: 0.698219120207268,

Versions

  • node-nlp 4.22.1
  • Node version v14.17.3

reinoute avatar Jul 20 '21 19:07 reinoute

I have tried the same with node-nlp 4.24.0 using both Node 12.18.2 and 16.14.2 and encountered the same issue. If I change the utterance being processed to "This thing I am saying is garbage and you should not be able to interpret it at all" it comes back with:

{ locale: 'en', utterance: 'This thing I am saying is garbage and you should not be able to interpret it at all', settings: undefined, languageGuessed: false, localeIso2: 'en', language: 'English', nluAnswer: { classifications: [ [Object], [Object] ], entities: undefined, explanation: undefined }, classifications: [ { intent: 'greetings.bye', score: 1 }, { intent: 'greetings.hello', score: 0 } ], intent: 'greetings.bye', score: 1, domain: 'default', sourceEntities: [], entities: [], answers: [ { answer: 'Till next time', opts: undefined }, { answer: 'see you soon!', opts: undefined } ], answer: 'see you soon!', actions: [], sentiment: { score: -1.501, numWords: 18, numHits: 6, average: -0.08338888888888889, type: 'senticon', locale: 'en', vote: 'negative' } }

I tried this because I have a model that works fine as long as the user gives it an expected input, but unexpected inputs always return one specific incorrect intent, often with a high degree of certainty.

AaronLRoberts avatar May 10 '22 19:05 AaronLRoberts

Hi, the None intent is a feature that needs to be activated in the nlu-manager. Try this:

const manager = new NlpManager({
  languages: ["en"],
  forceNER: true,
  nlu: {
    useNoneFeature: true,
  },
});

when instantiating the NlpManager, it should enable the use of the None intent when creating the NluManager:

{
  locale: 'en',
  utterance: 'This thing I am saying is garbage and you should not be able to interpret it at all',
  settings: undefined,
  languageGuessed: false,
  localeIso2: 'en',
  language: 'English',
  nluAnswer: {
    classifications: [ [Object] ],
    entities: undefined,
    explanation: undefined
  },
  classifications: [ { intent: 'None', score: 1 } ],
  intent: 'None',
  score: 1,
  sourceEntities: [],
  entities: [],
  answers: [],
  answer: undefined,
  actions: [],
  sentiment: {
    score: -1.501,
    numWords: 18,
    numHits: 6,
    average: -0.08338888888888889,
    type: 'senticon',
    locale: 'en',
    vote: 'negative'
  }
}

aigloss avatar Nov 24 '22 13:11 aigloss