Jarvis
Jarvis copied to clipboard
sukeesh
No natural commands and responses of the Assistant
I've been trying the assistant for a while and I notice that you need to be too specific about how to ask or tell things to the assistant, to the point that it's not natural to ask him or her. I think we could try to improve that. Let me know.
Yes, the language parser is far from perfect. It is just a search "is word XXX in input"? Unfortunately good language processing is not easy and something I have no experience at. Took a brief look at nltk (#348), but that's it. We had natural language topic already in #370 - but didn't hear anything since. So basically:
I think we could try to improve that.
Any suggestions welcome ;).
We could try to do some data science with pandas. I don't know much but I could learn in the go. I'm here to learn :grinning:
pandas
There are in fact libraries specially (and only) for natural language analysis. e.g. nltk.
Pandas is for data analysis - like if you measured temperature etc. every day of one year you could use pandas to plot a curve.
I don't think that's much use for us.
Yeah I got confused with something that I saw in a conference. What you are saying it's true. Let me investigate about nltk.
WOW, this luck:
Todays givaway by packt (official giveaway by publisher): "Next Generation Natural Language Processing with Python [Video]"
https://www.packtpub.com/packt/offers/free-learning
You would need to register and claim this video today (16 hours left) - but then its yours to keep. Looks quite interesting to me ;).
Just to ping @sukeesh @StefanosChaliasos just in case you are interested in this Natural Language Processing Video (because might be useful for this project and only free today). If not, please ignore ;).
I just saw it today :( I've been learning some things these days but at the same time getting confuse trying to understand the code of Jarvis.
I just saw it today :(
I don't even know if those tools are right for us - guess I was more excited than it actually was - so please don't mind ;).
at the same time getting confuse trying to understand the code of Jarvis
Understand; the core code uses some bit more "advanced" Python features and some meta-programming. This whole plugin.py and PluginManager.py code isn't that important for us here. It's just some code necessary to make creating commands easier and - I hope - just "nice".
In jarviscli/Jarvis.py - find_action(self, data, actions). That's the relevant part of the code.
data is the string entered by the user. e.g. 'roll a dice' actions is a list of available command names. e.g. ['battery', 'decrease', 'roll', ...] return String: COMMAND_NAME REMAINING_INPUT_PASSED_TO_PLUGIN
This method is called recursively (note: Just found a stupid bug related to this code so don't wonder that Jarvis crashes; I'm already fixing ;) #433)
Imagine you type: 'jarvis could you check my ram please'
find_action is called with 'jarvis could you check my ram please' find_action finds 'check' and returns 'check my ram please' (so check and everything after check)
find_action is called with 'my ram please' and ['forecast', 'time', 'weather', 'ram'] since that's the commands supported by 'check' find_action returns 'ram please'
The plugin 'check_ram' is called with 'please'.
Hop that makes stuff bit clearer ;). Of course we could re-write how find_action behaves if we need it. But if we could just rewrite the method find_actions to return more accurate results (and leave every other code untouched), I guess that could actually work...
PS: If anything is unclear about the code, please ask. I'm still learning, too and understanding how others read read code also helps me writing better, more understandable code ;).
NLP looks super interesting. Definitely, we can try with a library first. @pnhofmann @Zml99
@pnhofmann, thanks a lot, now the things are starting to get a little bit clearer. Next time I will not be afraid to ask.
Hey, just found really (!) interesting python library by chance:
https://github.com/snipsco/snips-nlu
I haven't tried it out yet. But basically: It's a complete machine-learning library which takes a sentence and returns correct command + parameter. All we would have to do is to produce the training data.
I think this is actually something we could use without bigger problems.
That's great. I've been busy with the university lately and I haven't been working in this for a while but this idea sound great.
Hey, I was looking at the code and I saw someone has been working into some nlp stuff. That's great! (Was it you @pnhofmann??) But as far as I can see it's pretty basic still. I decided now to start working hard into this, so you will see me more around here. Keep up the good work guys :)
Hey, I was looking at the code and I saw someone has been working into some nlp stuff.
Actually - not really... Just used nltk for dictionary-functionality if that is what you mean. Sorry ;).
Btw, someone know how can I make a branch in this project??
That's something I can do.
https://github.com/sukeesh/Jarvis/tree/natural_language
Snips-NLU looks great, and completely fulfills the purpose. Can't we change the language parser and integrate Snips with it ???
Can't we change the language parser and integrate Snips with it ???
That would be the idea ;). And I guess, this would actually be something possible with passable effort.
However work is still necessary. Mainly to generate the json traning data. You want to help ;) ?
I usually work with NLP.js, is a library for Javascript that gives you the opportunity to make natural language processing with more or less good results(they will be better with future training rounds). This library does not look for a exact word in a phrase, but makes a comparision between users input and a bunch of phrases that it has inside the training set and if it finds something similar gives the user the most similar output (or launch the function attached to that output). So, for example, if you say "Jarvis, could you please put some music" and inside the train you have phrases like "Turn on the music","I want rock songs" or "please,Jarvis...music" and the output of all this phrases is a function that launch the music inside a folder, the NLP will take the words [Jarvis, could, you, please, put, some, music ] in the train set and after compare with the array [ Turn, on, the,music,I, want, rock, songs, or,please,Jarvis,music ] if it finds many matches it will execute the function for this output. As you suppose the process is more complex and have another steps, but I do not have time for a longer explanation. At https://github.com/sukeesh/Jarvis/pull/526 I proposed a new merge that adds Node to Jarvis and give us the opportunity to try what we can do with NLP.js. Why Javascript and not Python for this?? just for my experience at work, I think that Python is great for many things, but around natural language processing, Javascript is much better and easier to implement.
Hi guys, I don't know if this is of any use, but I happened to find this:
https://github.com/nate-parrott/commanding
It's not as sophisticated as I imagine some of the machine learning options would be but it could be a good intermediate step while the proper training data for machine learning is generated. I tested it with a single command as its training data (translation, since I've been working on that a bit) and it is already better at picking out the source language, destination language and the phrase to be translated than anything I had come up with before.
Do we really need to go for Javascript?
I work with python and javascript and at this moment, the best NLP libraries that I could find are for the last one. I'm working in an implementation between Jarvis and NLP.js in a local repo, and the thing looks good. When I will finish this job I will update my repo, so if anyone want to try this option, be my guess and then, we can talk about if javascript gives us more problems that solutions. On the other hand,adding a new language to the project can bring us a new bunch of coders (with new ideas) that maybe are not great at python but in javascript.
Really I prefer python instead of javascript, but for NLP the second wins, in my opinion.
Hi, I would like to contribute to this. For example, a user who wishes to know what Jarvis can do, may ask 'what can you do?' or 'tell me what you can do' (i.e in a more natural way). I would make Jarvis be able to recognise these.
Also, while installing Jarvis, I get a message saying please install python3 but my pc already have python3. Somebody please help!
Hi @Karthick-Balaji! Sure, any contribution is welcome!
Also, while installing Jarvis, I get a message saying please install python3 but my pc already have python3.
Maybe open an issue for this one?
This shouldn't happen; what operating system are you running? If Linux / Mac what's the output of which python3?
My guess, could it be, that Python 3 isn't called 'python3' on your system, but something like e.g. 'python3.7' (and no symlink exists to python3)? I have seen behavior like that on FreeBSD; maybe I should add a routine to search for any executable that starts with python3.
Hey even I would like to contribute to this checkout wit.ai with this we can design a simple chatbot with that!!
@Karthick-Balaji installation issue now tracked here: #664
@chekoduadarsh Hey, Sorry - don't really like the idea of depending on a proprietary online service. Yes, we do integrate multiple online service for some features, but for something as essential as command parsing I would prefer an offline + Open Source approach.
I can add an Machine Learning approach to this voice assistant..... The assistant can predict the COVID-19 results such as deaths, recoveries, infections after one week beforehead.... You can watch my model on that...on-https://github.com/rishav-learnerml/Corona-Research also I had built a desktop assistant IRONMAN before a long time... You can watch it at - https://github.com/rishav-learnerml/Ironman the assistant will speak out the outcomes in voice and shall show the graphs and diagrams step by step.
Hi @rishav-learnerml !
this issue actually meant to recognize existing commands in a more natural way. Like not only type 'weather forecast' but something like 'do I need an umbrella'. Currently Jarvis searches for keywords (like translate or umbrella) but improvements could be made.
Really appreciate offer, but would rather not add stuff specially related to corona research, because I don't think that is the point of the Jarvis-project.
Will take a look at ironman.py, though.
I'm sorry that I didn't understand the motive... But yeah... As you said that you want to add more natural commands it is however easier to be done... You can check my Ironman project where I've made such a few natural commands... Those can easily be done.... I can add those to your project if you want :D
On Wed, 15 Apr 2020, 01:08 pnhofmann, [email protected] wrote:
Hi @rishav-learnerml https://github.com/rishav-learnerml !
this issue actually meant to recognize existing commands in a more natural way. Like not only type 'weather forecast' but something like 'do I need an umbrella'. Currently Jarvis searches for keywords (like translate or umbrella) but improvements could be made.
Really appreciate offer, but would rather not add stuff specially related to corona research, because I don't think that is the point of the Jarvis-project.
Will take a look at ironman.py, though.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sukeesh/Jarvis/issues/424#issuecomment-613641840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO6T4ZQPR4UXF5FIQXX5TXTRMS3TBANCNFSM4G2FCLVQ .
Hello there, If I may, I would like to introduce you to my F.R.I.D.A.Y project, which uses the NLP from wit.ai. I don't want to displease anyone here, but it would be great if you could contribute a bit to my new project, I admire your plugin technology e.g. very! Here's Friday: https://github.com/Tristi-X/F.R.I.D.A.Y
Snips-NLU has been implemented and working since a long time, at least in dev. I can see it understanding the correct command most of the times even when I give a confusing natural statement.
The way I see it, we will have to change the way we arrive at s in our code in order to be able to fully utilize it.
