ovos-core icon indicating copy to clipboard operation
ovos-core copied to clipboard

CommonQuery "tell me more"

Open JarbasAl opened this issue 1 year ago • 0 comments

proposal:

  • use the new streaming api from solver plugins, that yields utterances
  • regular common_qa skills can return a list of utterances instead of a single one
  • CommonQA pipeline needs to track "active skill", in order to speak the follow up utterances
    • done via Session - set_context("TellMeMore", skill_id) - to support voice sats
  • "tell me more" intent using context keywords, making that intent available (NOT a full enable/disable intent)
  • new class in ovos-workshop, CommonQA + Solver combination
    • this class only specifies the solver, the common_qa abstract methods are implemented in workshop, making it more friendly for skill devs
    • helper method to extract search keywords from utterance, KeywordExtraction plugins already exist and are used in some solvers individually, centralize main plugin in a single configurable place

this introduces an easy way for common query skills to have a shared continuous conversation component, "tell me more" to elaborate on the subject of previous answer

skills would need to handle the follow up speech when common query forwards it, a new bus message would be needed

Streaming Solvers

the default solver base class should handle stuff like wikipedia returning several paragraphs, advanced plugins can override this and provide real streaming for performance (like LLMs)

    def stream_utterances(self, query: str,
                          context: Optional[dict] = None) -> Iterable[str]:
        """streaming api, yields utterances as they become available
        each utterance can be sent to TTS before we have a full answer
        this is particularly helpful with LLMs"""
        ans = self.get_spoken_answer(query, context)
        for utt in self.sentence_split(ans):
            yield utt
d = WikiSolver()

query = "who is Isaac Newton"

for sentence in d.stream_utterances(query):
    print(sentence)
    # Sir Isaac Newton was an English mathematician, physicist, astronomer, alchemist, theologian, and author widely recognised as one of the greatest mathematicians and physicists of all time and among the most influential scientists.
   
    # He was a key figure in the philosophical revolution known as the Enlightenment.

    # His book Philosophiæ Naturalis Principia Mathematica, first published in 1687, established classical mechanics.
   
    # Newton also made seminal contributions to optics, and shares credit with German mathematician Gottfried Wilhelm Leibniz for developing infinitesimal calculus.
  
    # In the Principia, Newton formulated the laws of motion and universal gravitation that formed the dominant scientific viewpoint until it was superseded by the theory of relativity.

JarbasAl avatar Feb 22 '24 15:02 JarbasAl