Adapt won't catch correctly

[whtecmp]

Hi, I was working on developing a skill when I encountered this issue: my init.py file is:

from mycroft import MycroftSkill, intent_handler
from adapt.intent import IntentBuilder
from mycroft.skills.context import removes_context, adds_context

class Debug(MycroftSkill):
    def __init__(self):
        MycroftSkill.__init__(self)

    @intent_handler(IntentBuilder('Search').require('debug_details'))
    @adds_context('SearchResultsContext')
    def handle_debug(self, message):
        self.set_context('package_name', 'python')     
        self.speak('Details are {}'.format(message.data.get('debug_details')))
        self.speak('Uterrance was {}'.format(message.data.get('utterance')))
        
def create_skill():
    return Debug()

I have a single file in locale/en-us/ called other.rx whose content is: .* debug (?P<debug_details>.+) Now, I do this query: find debug python only if begins with python, twice and here is the dialog:

 find debug python only if begins with python                             
 >> Details are python only if begins with python                         
 >> Uterrance was find debug python only if begins with python            
 find debug python only if begins with python
 >> Details are python only if begins with
 >> Uterrance was find debug python only if begins with python

The expected result is to get the same response twice, but for some reason the second response lacks the final word python. I boiled down the issue to the line: self.set_context('package_name', 'python')

I did all of this in the Mycroft cli, on Debian 10. Mycroft version 20.8.0 (branch master) I also attached a tar.gz file containing a skill with this issue. debug.tar.gz

[forslund]

Thanks for reporting and providing the test case. I'll try to replicate and see if it's an adapt issue or if it's the Mycroft's implementation of the context stack that acts up here. It's likely an issue with the regex confidence scaling compared with the context confidence scaling.

[forslund]

Ok tried it and this does look really weird...I can get the issue without the context changing code so I wonder if it's adapt's regex handling that's the root cause?

[whtecmp]

Did you just remove the self.set_context('package_name', 'python') to replicate the effect without context? Because for me the issue doesn't happen without this line. If you did something else, could you send it here, so I'll be able to try and debug this too?

[whtecmp]

Looking at message.data more closely, I noticed another wired thing. I compared the first time I ask the query to the second time. The first time the confidence is 0.125 the second time it is 0.0625. If I run adapt manually (code later) with the same IntentBuilder I get 0.25 confidence.

engine = IntentDeterminationEngine()
regex = '.* debug (?P<debug_details>.+)'
intent_filter = IntentBuilder('Search').require('debug_details').build()
engine.register_intent_parser(intent_filter)
engine.register_regex_entity(regex)
for intent in engine.determine_intent('find debug python only if begins with python'):
    print (intent)
    break

[forslund]

I can replicate "some" weirdness without any of the lines setting context. I basically get nothing after the Details are if I use the example sentence. If I remove the second "python" it works flawlessly

Adapt confidence is a bit wonky especially where the regex part is concerned. But I think it gets lower the more possible matches there can be...or something similar. @clusterfudge knows the ins and outs of adapt the best and might be able to tell us how it really works :)

I haven't had time to experiment with this just yet but I won't rule out the context being a factor.

[JarbasAl]

context should decrease confidence i think, the older it is the less confidence it will match with.

If the keyword is in the utterance and not injected by context then it should always be the same (assuming no new intents registered meanwhile)

[forslund]

The weird thing in the debug example is that the set context isn't the same keyword as is used in the regexp.

[oblitum]

I think I'm facing similar issues. I'm trying to have optionally regex-based intent parts in an intent, but anytime I do that I never receive the optional fragment in message.data, even when it's present in the utterance. But if I just change optionally to require, that part is captured in message.data. I simply can't figure out what's going on.

~~Given that I can't figure out how to make "optionally" work, my only option is emulate optional parts by having two intent handlers, one simply without the given part to look for, and the other with it required.~~ This doesn't work as the shorter intent always matches and overrides the longer one.