deep_qa
deep_qa copied to clipboard
evaluate_model() gives parsing error
I am trying to evaluate a model using evaluate_model()
evaluate_model("some.json")
but it gives a ParseException.
some.json-
{
"model_class": "BidirectionalAttentionFlow",
"model_serialization_prefix": "/models/bidaf",
"encoder": {
"word": {
"type": "cnn",
"ngram_filter_sizes": [5],
"num_filters": 100
}
},
"seq2seq_encoder": {
"default": {
"type": "bi_gru",
"encoder_params": {
"units": 100
},
"wrapper_params": {}
}
},
"data_generator": {
"dynamic_padding": true,
"adaptive_batch_sizes": true,
"adaptive_memory_usage_constant": 440000,
"maximum_batch_size": 60
},
"patience": 3,
"embeddings": {
"words": {
"dimension": 100,
"pretrained_file": "/trained_vectors/glove.840B.300d.txt",
"project": true,
"fine_tune": false,
"dropout": 0.2
},
"characters": {
"dimension": 8,
"dropout": 0.2
}
},
"num_epochs": 20,
"optimizer": {
"type": "adadelta",
"learning_rate": 0.5
},
"validation_files": ["dev_squad.json"],
"train_files": ["train_squad.json"]
}
This is the stacktrace-
Using TensorFlow backend.
processor = {}
processor.word_splitter = simple
processor.word_filter = pass_through
processor.word_stemmer = pass_through
0 1175
Traceback (most recent call last):
File "D:/backup/PycharmProjects/test/deep_qa/with_qa.py", line 79, in <module>
evaluate_model("some.json")
File "D:\backup\PycharmProjects\test\deep_qa\run.py", line 214, in evaluate_model
model = load_model(param_path, model_class=model_class)
File "D:\backup\PycharmProjects\test\deep_qa\run.py", line 147, in load_model
param_dict = pyhocon.ConfigFactory.parse_file(param_path)
File "C:\Users\mohit.badwal.NOTEBOOK546\Anaconda3\lib\site-packages\pyhocon\config_parser.py", line 51, in parse_file
return ConfigFactory.parse_string(content, os.path.dirname(filename), resolve)
File "C:\Users\mohit.badwal.NOTEBOOK546\Anaconda3\lib\site-packages\pyhocon\config_parser.py", line 90, in parse_string
return ConfigParser().parse(content, basedir, resolve)
File "C:\Users\mohit.badwal.NOTEBOOK546\Anaconda3\lib\site-packages\pyhocon\config_parser.py", line 272, in parse
config = config_expr.parseString(content, parseAll=True)[0]
File "C:\Users\mohit.badwal.NOTEBOOK546\Anaconda3\lib\site-packages\pyparsing.py", line 1216, in parseString
raise exc
File "C:\Users\mohit.badwal.NOTEBOOK546\Anaconda3\lib\site-packages\pyparsing.py", line 1210, in parseString
se._parse( instring, loc )
File "C:\Users\mohit.badwal.NOTEBOOK546\Anaconda3\lib\site-packages\pyparsing.py", line 1072, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "C:\Users\mohit.badwal.NOTEBOOK546\Anaconda3\lib\site-packages\pyparsing.py", line 2545, in parseImpl
loc, exprtokens = e._parse( instring, loc, doActions )
File "C:\Users\mohit.badwal.NOTEBOOK546\Anaconda3\lib\site-packages\pyparsing.py", line 1076, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File "C:\Users\mohit.badwal.NOTEBOOK546\Anaconda3\lib\site-packages\pyparsing.py", line 2348, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pyparsing.ParseException: Expected end of text (at char 0), (line:1, col:1)
Where am I going wrong ?
I put that file into https://jsonlint.com/, and it said it was valid json. I'm not sure why pyhocon is complaining when parsing it. When I copy that file and run evaluate_model("some.json")
on my linux machine, it also parses (it then crashes because it doesn't have write permission on /models, but that's a different issue).
My best guess is that pyhocon behaves differently with respect to line endings or something on windows, so it isn't parsing the file correctly. Also, I'm pretty sure there are plenty of parts of this code that will break when running on windows. See #335.