Chatbot
Chatbot copied to clipboard
JRC1995
Hello I am having a problem with the database and setup
Hello I am having a problem with the database and setup specifically get large index and then the reddit db. do you know how that would work and where i could get them or set them up, also.
Just wondering doesthe project use Chatterbot.corpus or chatterbot itself.
So I am having troubles with setting up the reddit.db, I have a faiss_index/large.index error where it wont open it says error it didn't fail cause 1 doesnt equal zero and I also have a very big float problem sadly and quite a few more problems in the meta querry encoder, just wondering do you know how to solve them, I can detail more specifically, which lines and etc if you would like ig.
In addition in interact.py Error in __cdecl faiss::FileIOReader::FileIOReader(const char *) at D:\a\faiss-wheels\faiss-wheels\faiss\faiss\impl\io.cpp:68: Error: 'f' failed: could not open Retriever/Faiss_index/large.index for reading: No such file or directory
File "C:\Users\Colin Chau\Desktop\poly ai\Chatbot-master\interact.py", line 45, in
No I am not aware of those specific problems. I don't remember facing them. You can share the errors one at a time and I can check them. First, we can start with reddit.db setup issue. IIRC, reddit.db is setup before creating faiss. So is there an issue with the initial setup of reddit db. Is it the issue with meta_query_encoder?
On Thu, Sep 30, 2021, 11:58 AM IshFire123 @.***> wrote:
So I am having troubles with setting up the reddit.db, I have a faiss_index/large.index error where it wont open it says error it didn't fail cause 1 doesnt equal zero and I also have a very big float problem sadly and quite a few more problems in the meta querry encoder, just wondering do you know how to solve them, I can detail more specifically, which lines and etc if you would like ig.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JRC1995/Chatbot/issues/4#issuecomment-931499508, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYBQNXGKR3ENH5KRFQCA3LUESJLBANCNFSM5E6KRERA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
In meta query encoder, there's a problem line 43,says float wont work the float32 variable is supposedly the problem but I can't seem to find a fix.
That's odd. Are you using the same environment as in the yaml file? Also do you have access to ConveRT (someone notified that it is not public anymore)? Do you have the pretrained weights downloaded?
On Thu, Sep 30, 2021, 4:00 PM IshFire123 @.***> wrote:
In meta query encoder, there's a problem line 43,says float wont work the float32 variable is supposedly the problem but I can't seem to find a fix.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JRC1995/Chatbot/issues/4#issuecomment-931683644, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYBQNWA5Y2ZGVNFN5NKXATUETFWDANCNFSM5E6KRERA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
Expects arg[0] to be string but float is provided [[node StatefulPartitionedCall_1 (defined at c:\users ok thats thproblem i think lemme figure the rest out i guess Exception has occurred: InvalidArgumentError Expects arg[0] to be string but float is provided [[node StatefulPartitionedCall_1 (defined at c:\users\Person\desktop\poly ai\chatbot-master\Sentence_Encoder\meta_query_encoder.py:41) ]]
Errors may have originated from an input operation. Input Source operations connected to node StatefulPartitionedCall_1: QA/Final/Response_tuning/ResidualHidden_1/dense/kernel (defined at C:\Users\Person\anaconda3\envs\Chatbotintents\lib\site-packages\tensorflow_hub\module_v2.py:106)
Original stack trace for 'StatefulPartitionedCall_1':
File "C:\Users\Person\anaconda3\envs\Chatbotintents\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\Users\Person\anaconda3\envs\Chatbotintents\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "c:\Users\Person.vscode\extensions\ms-python.python-2021.9.1246542782\pythonFiles\lib\python\debugpy_main.py", line 45, in
During handling of the above exception, another exception occurred:
File "C:\Users\Person\Desktop\poly ai\Chatbot-master\Sentence_Encoder\meta_query_encoder.py", line 42, in encode
USE_QA = np.asarray(sess.run(USE_QA_encoder), np.float32)
File "C:\Users\Person\Desktop\poly ai\Chatbot-master\Scripted\Subscripts\process_pkl.py", line 37, in process
embedded_chatterbot_queries = encode(sess,chatterbot_queries,chatterbot_contexts,USE_model, ConvRT_model)
File "C:\Users\Person\Desktop\poly ai\Chatbot-master\Scripted\setup.py", line 12, in
That's odd. Are you using the same environment as in the yaml file? Also do you have access to ConveRT (someone notified that it is not public anymore)? Do you have the pretrained weights downloaded? … On Thu, Sep 30, 2021, 4:00 PM IshFire123 @.***> wrote: In meta query encoder, there's a problem line 43,says float wont work the float32 variable is supposedly the problem but I can't seem to find a fix. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYBQNWA5Y2ZGVNFN5NKXATUETFWDANCNFSM5E6KRERA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
I think I have I am pretty sure do you have the files just wondering.
This is the current problem but i fixed it by doing this
Could this be the problem
Also does Reddit.db have to be created in sqlite or downloaded just wondering.
Never mind I found a fix. In that case before I continue, do I need to put anything in the CSV files or such.
Yes the csv files were generated from google query. I retrieved reddit data from there. https://pushshift.io/using-bigquery-with-reddit-data/ https://towardsdatascience.com/scrape-reddit-data-using-python-and-google-bigquery-44180b579892 There are also some descriptions here: https://github.com/JRC1995/Chatbot#component--1-scripted-module
Can you elaborate?
On Fri, Oct 1, 2021, 12:52 PM Colin Chau @.***> wrote:
Ok I found the problems, Its jsut that now I can't seem to get Fill data to fill in any data and then im still missing the index file I guess.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JRC1995/Chatbot/issues/4#issuecomment-932437178, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYBQNRRCHERLXTS4G3TA2DUEXYPFANCNFSM5E6KRERA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
Also does Reddit.db have to be created in sqlite or downloaded just wondering.
I used SentenceTransformer instead of USE_QA and ConveRT and I was able to solve the encoding problem. But I do need to know how can I get access to the reddit.db and large.index files in faiss directory ?? Thanks in Advance
It should be according to the instructions. After you have the csv files, you can create the reddit.db by running fill_data.py and then create the index by running faiss_it.py from here: https://github.com/JRC1995/Chatbot/tree/master/Retriever But before running them you should make sure that every place consistently uses the same embeddings that you using: https://github.com/JRC1995/Chatbot/search?q=USE_QA