Demo Server and Inference Module

[webeng]

• Updated the demo branch so it's now a standalone module. It accepts new text and outputs answers and soft-maxed logits (confidence).
• Added an inference module to predict answers given any context and query.
• Added a .gitignore file for people who work with virtualenv
• Updated requirements.txt to use the right tf version (0.12.1)
• Added comments to squad.prepro
• Added a function to squad.prepro to preprocess a single context and query.
• Updated README to explain how to use the demo and inference module.

[VishalVaddina]

Hey Joan, Thanks for the inference module. I've tried it out and found an issue you might wanna know.

During inference, for the same context and question, the answer sometimes returns a string and sometimes returns {'56be4db0acb8001400a502ee': 7035271525382996}.

I've tried this on many multiple instances, many times to confirm. And every time the problem has been same. Under same environment, same code, same seed, for same context and question pair, the results vary randomly. Approximately 3/10 times, I was getting correct answer and for the rest of the times I was getting {'56be4db0acb8001400a502ee': 0.7035271525382996}.

See the pic below for example:

I've checked your code and solved this issue by commenting out lines 388 to 390 in basic/evaluator.py in your master branch, and return values from copies of id2answer_dict and id2score_dict respectively as answer and confidence score.

I've read that this issue is due to atomicity in Python-3, which causes an exception when the dictionary is modified at the same time by another thread. Reference: https://blog.labix.org/2008/06/27/watch-out-for-listdictkeys-in-python-3

I don't know if you've ever faced this issue before. But in case you did, the inference works perfectly with above modifications.

Thanks.