how to predict answers for custom question and context by reusing loaded model.

[rubby33]

Hi: After i reload the mrc model, how can I implement the function like predict to get the answers for different input questions and contexts.

Meanwhile, i want to reuse the loaded mrc model and session. I do not want to reload model and initialize the session many times. Thanks

[rubby33]

because graph_handler depends on the model from get_multi_gpu_models(), while the model from get_multi_gpu_models() depends on the input questions and contexts.

[seominjoon]

Could you tell me what "mrc model" is?

[rubby33]

"mrc model" i mean the the restored the model by "saver.restore(sess, save_path)" . My question is :
I have many json files whose format are the same to the standard train file or dev file, but in each of my json file, there is only one paragraph, one question. I want to use the bi-att-flow(mrc model) to predict the answer.

Since the following scrip will cost so much time:

basic/run_single.sh $HOME/data/squad/mytest1.json single.json basic/run_single.sh $HOME/data/squad/mytest2.json single.json ... basic/run_single.sh $HOME/data/squad/mytestn.json single.json

Can i use a script(e.g. python file) to restored the model and initialize everything once, then i can use it on different json files? which is like:

result =pythonscript.predict("mytest1.json") print ("the answer for mytest1.json is ", result) result =pythonscript.predict("mytest2.json") print ("the answer for mytest2.json is ", result)

or, result =pythonscript.predict(context,question) print ("the predict result is: ", result) result =pythonscript.predict(context2,question2) print ("the predict result is: ", result)

I think this will bring some convenience for test

Thanks for your reply

[seominjoon]

Basically you want to set up a server which can take in question and output prediction, is that correct? As you pointed out, calling script every time will be costly. At this point, probably the easiest thing without touching anything is use our demo. You can host your own demo by using demo branch (https://github.com/allenai/bi-att-flow/tree/demo). Then you can perform GET request to directly get the answer from the server.

[rubby33]

That's what I need. Thanks a lot. Let me have a try. 👍

[salmedina]

@rubby33 Were you able to run the demo?

I tried to run it and faced the problem that the pretrained model at out/basic/00/save/basic-18000 wasn't provided and instead there is a file named basic-20000 under the same location.

However, now I face a new problem while trying to run python run-demo.py, the program throws a:

NotFoundError (see above for traceback): Tensor name "model_0/prepro/u1/BW/BasicLSTMCell/Linear/Matrix/ExponentialMovingAverage" not found in checkpoint files out/basic/00/save/basic-20000

is there anything else that I should set into the running configuration? Thank you

[bluelight773]

Same issue here. Thank you.

[webeng]

Are you running on tensorflow < 1.0?

[bluelight773]

yes, installed r0.11 to match the readme.tensorflow.__version__gives me 0.11.0.

Not sure if this is the issue but the following subfolders inside out/basic/00 are empty for me right now: eval, answer, log.

[webeng]

Ohhh I see, you're running on the demo branch right?

If I remember rightly, I copied all the flags in basic/cli.py and replaced all the flags in basic/demo_cli.py for the ones in cli.py, then I changed the following flags to pick the data from the correct location:

flags.DEFINE_string("data_dir", "path-to-bi-att-flow/data/squad", "Data dir [data/squad]") flags.DEFINE_string("eval_path", "data/squad", "Eval path []") flags.DEFINE_string("load_path", "path-to-bi-att-flow/save/37/save", "Load path []") flags.DEFINE_string("shared_path", "path-to-bi-att-flow/save/37/shared.json", "Shared path []") flags.DEFINE_string("mode", "forward", "trains | test | forward [test]")

You can also change the flags on the command line.

The demo branch is experimental...

I hope it helps.

[bluelight773]

Thank you! I ran: python run-demo.py --data_dir=data/squad --eval_path=data/squad --load_path=save/37/save --shared_path=save/37/shared.json --mode=forward and got the server running.

This is perhaps a different matter in need of a separate issue (please advise): In the online demo, we're able to write our own paragraph and ask a question about it (as opposed to only select one of the predefined paragraphs). The UI on the demo running on my machine doesn't seem to allow writing my own paragraph. Is this possible somehow? Is there perhaps some curl command I could run passing in a paragraph and a question and getting an answer in response?

Apologies in case I'm missing something obvious.

Thank you

[webeng]

Yeah!

The demo only works with text from the dataset. If you look into the code, the client is only sending the paragraph_id instead of the text itself so even if you change the text, the backend will go and pick the original paragraph.

I hope it helps.

On 6 Apr 2017, at 20:54, Norris [email protected] wrote:

Thank you! I ran: python run-demo.py --data_dir=data/squad --eval_path=data/squad --load_path=save/37/save --shared_path=save/37/shared.json --mode=forward and got the server running.

This is perhaps a different matter in need of a separate issue (please advise): In the online demo, we're able to write our own paragraph and ask a question about it (as opposed to only select one of the predefined paragraphs). The UI on the demo running on my machine doesn't seem to allow writing my own paragraph. Is this possible somehow? Is there perhaps some curl command I could run passing in a paragraph and a question and getting an answer in response?

Apologies in case my question exposes me missing some obvious matter.

Thank you

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/allenai/bi-att-flow/issues/13#issuecomment-292296166, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfBQRHTJ0EC6VFhzjM-KZdRnpPvCsr7ks5rtULhgaJpZM4Mn3xU.

[seominjoon]

@webeng Hmm, in fact, it seems the demo branch is not up to date.. It used to be like that, and we changed it later (current demo). @shmsw25 Could you please take a look at this?

[nikhilaravi]

@bluelight773 I tried running your command and the server now works but when I press submit on a question I can an internal server error. Did you also experience this problem?

@shmsw25 I'm also interested in adding my own paragraphs - would you be able to update the demo branch to allow this?

[bluelight773]

@nikhilaravi I'm not getting the server error you're getting. When I submit one of the pre-defined questions on one of the pre-defined paragraphs (which seems to be all that's supported in the current version of the demo on the repo), I get an answer as expected. Not sure how our set-ups may differ. For what it's worth, I'd set up a conda environment where I ensured the required packages listed in the readme here are installed with the specified versions (where mentioned): https://github.com/allenai/bi-att-flow/blob/master/README.md.

@shmsw25 I second @nikhilaravi 's request on updating the branch if possible :) I'd imagine many people would be interested in the functionality of specifying own paragraphs and own questions using the pretrained model as is already supported in the online demo. :)

[nikhilaravi]

@bluelight773 thanks I managed to fix the issue! @bluelight773 I managed to get user input of paragraphs working - just clone @shmsw25's fork of the repo and use the demo branch (from #22 I noticed it had been updated in her version and is just waiting to be merged into this repo)

[seominjoon]

Hi all, the demo branch has been updated. Now it can easily accept custom para/ques through http request.

[salmedina]

Tested, it works as the online demo! Thank you very much for the hardwork!

[rubby33]

Thanks，great work. It works fine after several test.

According the original shell test approach,such as "basic/run_single.sh $HOME/data/squad/dev-v1.1.json single.json". The following config parameters may change according to different custom paragraph and question. e.g., max_word_size depends on the custom paragraph and question.

   **config.max_sent_size = config.sent_size_th
    config.max_num_sents = config.num_sents_th
    config.max_ques_size = config.ques_size_th
    config.max_word_size = config.word_size_th
    config.max_para_size = config.para_size_th**

While in the Demo branch, these parameters are constant that are changeless. So would it have some negative effects on accuracy of the model???

Thanks!

[nikhilaravi]

@seominjoon @rubby33 @salmedina How can we return the top 5 answers for a question along with the scores instead of only the top answer?

[seominjoon]

@rubby33 If the sentence doesn't fit into the threshold, the sentence gets truncated from the end (if I remember correctly). So this will lose accuracy. Otherwise, the accuracy should be the same (i.e. larger threshold doesn't hurt). Of course, larger thresholds needs more GPU RAM.

@nikhilaravi I want to add this functionality in future, but for now probably the easiest way to do it is looking into the eval-XX.json (or eval-XX.pklz) files. How to use this file can be seen basic/ensemble.py of dev branch. https://github.com/allenai/bi-att-flow/blob/dev/basic/ensemble.py#L94 This line of code computes the best span, but you can sort it and get best 5 if you want to.

[nikhilaravi]

@seominjoon thanks very much, I'll look into it!

[diamondspark]

Hi, I have been trying to figure out why the model doesn't work so well for short sentences. For instance if my input paragraph is "I go to school" and the question is "Where do I go to" , I dont get an answer. http://squad.budianto.id which is an extension of your BiDaf seems to be able to work well for short sentences but performs poorly in comparison to yours for longer paragraphs. This makes me wonder if there is a window parameter which I can play around to make your Bi-Daf work better for smaller sentences.

[seominjoon]

@diamondspark This is bug actually, related to #18 Fixing this is easy, although I didn't update the master branch yet. For now, probably the easiest way around this is put a word such as period at the end (e.g. "I go to school." instead of "I go to school").

[bluelight773]

@seominjoon back to the question regarding thresholds, 1) can you confirm that when running the demo, that these are the thresholds that are in effect? (From basic/demo_cli.py) flags.DEFINE_integer("word_count_th", 30, "word count th [100]") flags.DEFINE_integer("char_count_th", 150, "char count th [500]") flags.DEFINE_integer("sent_size_th", 1000, "sent size th [64]") flags.DEFINE_integer("num_sents_th", 1000, "num sents th [8]") flags.DEFINE_integer("ques_size_th", 100, "ques size th [32]") flags.DEFINE_integer("word_size_th", 48, "word size th [16]") flags.DEFINE_integer("para_size_th", 1000, "para size th [256]")

Above values are different than what they are are in basic/cli.py

2. Could we change those values, in basic/demo_cli.py, before running the demo to any desired value and they'd take effect once the server is up and running?

3. Can you confirm, we don't actually need to train a new model with new threshold values. We can just update those values and they'd apply regardless of the model we're using?

4. I don't wanna make this too long, but if you're able to specify the meaning of those 7 parameters, it'd be quite helpful as for instance it's not obvious in some cases whether the numbers are referring to number of chars or words, or whether that number applies in the context of a word/sentence/paragraph.

Thank you

[seominjoon]

@bluelight773

1. The params are right. You can ignore num_sents_th because right now entire paragraph is considered as a single sentence. So basically, you can put up to 1000-word context into the demo, and you can increase it arbitrarily. Note that this will slow down in CPU and could result in OOM in GPU.

2. Yes, you can change them to any values before running the server and they will be effective right away.

3. Yes, those params can be arbitrarily changed without retraining.

word_count th, char count th: ignore these for now sent_size_th: maximum length (# of words) of each sentence num_sents_th: maximum number of sentences. This is not used now, because the entire paragraph is considered as a single sentence. ques_size_th: maximum # of question words word_size_th: length (# of chars) of each word. If exceeds, only take first 48 chars. para_size_th: maximum number of words in the context. Since the entire para is considered as one sentence, this should equal sent_size_th

[lbbc1117]

Does this demo have to be run on tensorflow 0.11 ?

[seominjoon]

@lbbc1117 Yes, but if you want to make it compatible to v1.2, perhaps the best way is to look at the dev branch, which is compatible with v1.2. Currently I am not sure of the ETA for when I would be able to make the demo compatible with v1.2..

[CriMenghini]

@rubby33 When you create the json files whose format are the same to the standard train file or dev file. Do you put, for each question, the answer_start parameter?

[smitar]

The quotes around the title name "News" in the json doesn't look right. There are a few other places in the json which has the wrong quotes.

Thanks, Smita

On Sat, Dec 2, 2017 at 6:12 PM, Bhavika Reddy Jalli < [email protected]> wrote:

I am trying to predict the answer to my own custom question using the pre-trained model. The test1.json file following the Squad format is as follows:

{"data": [{"title": “News”, "paragraphs": [{"context": "jordan 's crown prince hassan ibn talal arrived tuesday for his first visit to jerusalem and was to pay his condolences to the widow of assassinated prime minister yitzhak rabin .", "qas": [{"answers": [{"answer_start": 81, "text": “jerusalem”}, {"answer_start": 81, "text": "jerusalem"}, {"answer_start": 81, "text": "jerusalem"}], "question": “where did the crown prince visit?”, "id": “1”}]}]}], "version": "1.1"}

However, when I run the command basic/run_single.sh test1.json single.json, I get the following error

raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 21 (char 20)

Can someone help me out?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/allenai/bi-att-flow/issues/13#issuecomment-348734865, or mute the thread https://github.com/notifications/unsubscribe-auth/AHFlo_J7j5U_rCYirz8uRawK1cmBSRdTks5s8gN4gaJpZM4Mn3xU .