Issue with NLP manager Export-Import

[gargshantnu]

Describe the bug

If I train/build the NlpManager fresh from scratch with some fixed training phrases, it is working fine ( the results are as expected ). But if i am exporting that agent into JSON file and importing the same, the results of .process method are different depending upon in which order the training phrases are added to the manager ( before exporting ).

To Reproduce

Test 1:

1. Run nlp-test-export-1.js. It will export the model and create an nlpModel.json file. Also check logs, the response1, response2 should have intent names as email.extraction
2. Run nlp-test-import.js. It will import the model, process the msg and log the response. Check logs, the response1, response2 should have intent names as None.

Test 2:

1. Run nlp-test-export-2.js. It will export the model and create an nlpModel.json file. Also check logs, the response1, response2 should have intent names as email.extraction
2. Run nlp-test-import.js. It will import the model and give log the response. Check logs, the response1, response2 should have intent names as email.extraction.

Conclusion: Bot output is depending upon the order in which training phrases are feeded in the model.

Files

nlp-test-export-1.js

const { NlpManager } = require('node-nlp');
let manager = new NlpManager({ languages: ["en"] });

// SET 1 - not working as expected
manager.addDocument('en', 'number 91919919119', 'number.extraction.test');
manager.addDocument('en', 'price', 'pricing');
manager.addDocument('en', 'pricing?', 'pricing');
manager.addDocument('en', 'what is your pricing?', 'pricing');
manager.addDocument('en', ',', 'random.stuff');
manager.addDocument('en', 'adad', 'random.stuff');
manager.addDocument('en', 'kalda', 'random.stuff');
manager.addDocument('en', 'contact me at %email%', 'email.extraction');
manager.addDocument('en', 'my email is %email%', 'email.extraction');

(async() => {
    await manager.train();

    const response1 = await manager.process('en', 'my email is [email protected]');
    const response2 = await manager.process('en', 'contact me at [email protected]');
    
    const minified = true;
    const nlpModel = manager.export(minified);
    const fs = require('fs');
    fs.writeFile('nlpModel.json', nlpModel, 'utf8', ()=>{});

    console.log(response1.intent);
    console.log(response2.intent);
})();

nlp-test-export-2.js

const { NlpManager } = require('node-nlp');
let manager = new NlpManager({ languages: ["en"] });

// SET 2 - working fine
manager.addDocument('en', 'number 91919919119', 'number.extraction.test');
manager.addDocument('en', 'what is your pricing?', 'pricing');
manager.addDocument('en', 'pricing?', 'pricing');
manager.addDocument('en', 'price', 'pricing');
manager.addDocument('en', 'my email is %email%', 'email.extraction');
manager.addDocument('en', 'contact me at %email%', 'email.extraction');
manager.addDocument('en', ',', 'random.stuff');
manager.addDocument('en', 'kalda', 'random.stuff');
manager.addDocument('en', 'adad', 'random.stuff');

(async() => {
    await manager.train();

    const response1 = await manager.process('en', 'my email is [email protected]');
    const response2 = await manager.process('en', 'contact me at [email protected]');
    
    const minified = true;
    const nlpModel = manager.export(minified);
    const fs = require('fs');
    fs.writeFile('nlpModel.json', nlpModel, 'utf8', ()=>{});

    console.log(response1.intent);
    console.log(response2.intent);
})();

nlp-test-import.js

const { NlpManager } = require('node-nlp');
let manager = new NlpManager({ languages: ["en"] });

(async() => {
    const nlpModel = require('./nlpModel.json');
    manager.import(nlpModel);

    const response1 = await manager.process('en', 'my email is [email protected]');
    const response2 = await manager.process('en', 'contact me at [email protected]');
    
    console.log(response1.intent);
    console.log(response2.intent);
})();

Expected behavior

Since Training phrases are same, .process method should give same results.

Desktop (please complete the following information):

Desktop (please complete the following information):

• OS: macOS
• Package version - "4.10.4"
• Node version - v12.18.4

Additional context

It was not working in my personal project, so after some debugging came to the conclusion that same training phrase added in different order are giving different results.

[jesus-seijas-sp]

Hello,

What is really happening: The real bug is that the slot filling is not loading correctly, when the slot filling is recovered it recovers the slots, but the property isEmpty is still true. So it does not generate the optional utterance. What is the optional utterance? When you provide a sentence like "my email is [email protected]" two utterances are generated: "my email is [email protected]" and "my email is %email%". With this, we have a signature to calculate exact sentences.

So I will fix this one, that is the bug.

But more things:

• Your corpus is too small and with no so many examples. In this case I recommend you to deactivate the useNoneFeature property

let manager = new NlpManager({ languages: ["en"], nlu: { useNoneFeature: false } });

Because your corpus is so small that this feature is causing noise. Or perhaps I should put it to false by default.

• Your corpus is using the intent random.stuff it the way that the default fallback intent (None) is defined, so you will have None and random.stuff fighting between them... I suggest to replace random.stuff with None

• About the order of the intents in the training: this is normal that different sort of items generates different weights on the neural network, in fact, almost every single Neural Network in the world will return to you different scores even training with the same data, because usually they initialize to gaussian noise.

[jesus-seijas-sp]

Hello, try updating to version 4.9.0, it solves the issue with slots and also set the useNoneFeature to false by default.

[gargshantnu]

Hello, Thanks for the suggestions. Both the solutions worked for me:

1. Removing random.stuff training phrases with some valid inputs.
2. keeping useNoneFeature: false.

Thanks for the help. It worked for me. Just 1 confusion is there, why it was breaking only in Export-Import and not when i was building a new model.

[gargshantnu]

Hello, try updating to version 4.9.0, it solves the issue with slots and also set the useNoneFeature to false by default.

@jesus-seijas-sp I hope here you are referring to 4.19.0 :)

[jesus-seijas-sp]

Hello, try updating to version 4.9.0, it solves the issue with slots and also set the useNoneFeature to false by default.

@jesus-seijas-sp I hope here you are referring to 4.19.0 :)

Yes, sorry, 4.19.0