Neural-Baby-Talk-python3 icon indicating copy to clipboard operation
Neural-Baby-Talk-python3 copied to clipboard

KeyError: 'type'

Open ghost opened this issue 5 years ago • 15 comments

I try your training code. But return the "KeyError: 'type'" Traceback (most recent call last): File "/home/gyq/workspace/nbt3/main.py", line 215, in dataset = DataLoader(opt, split='train') File "/home/gyq/workspace/nbt3/misc/dataloader_coco.py", line 118, in init self.coco_train = COCO(det_train_path) File "tools/coco-caption/pycocotools/coco.py", line 79, in init self.createIndex() File "tools/coco-caption/pycocotools/coco.py", line 96, in createIndex if self.dataset['type'] == 'instances': KeyError: 'type' How can I Slove this problem? It seems the "instance2014.json " don't have the key "type".

ghost avatar May 22 '19 12:05 ghost

I have a repo of modified coco-caption/ for python 3. Did you try it, clone it to replace your current coco-caption/ dir. Here: https://github.com/luulinh90s/coco-caption

Vào 21:54, T.4, 22 Th5, 2019 yizhiyua [email protected] đã viết:

I try your training code. But return the "KeyError: 'type'" Traceback (most recent call last): File "/home/gyq/workspace/nbt3/main.py", line 215, in dataset = DataLoader(opt, split='train') File "/home/gyq/workspace/nbt3/misc/dataloader_coco.py", line 118, in init self.coco_train = COCO(det_train_path) File "tools/coco-caption/pycocotools/coco.py", line 79, in init self.createIndex() File "tools/coco-caption/pycocotools/coco.py", line 96, in createIndex if self.dataset['type'] == 'instances': KeyError: 'type' How can I Slove this problem? It seems the "instance2014.json " don't have the key "type".

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/luulinh90s/Neural-Baby-Talk-python3/issues/4?email_source=notifications&email_token=AHLNPF2G4UC6KKEXVXWJ65LPWU7B5A5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GVGQISQ, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPFZRTDOPFJHWEQOKN2LPWU7B5ANCNFSM4HOUACBA .

giangnguyen2412 avatar May 22 '19 18:05 giangnguyen2412

Thank you for your patient reply. I have a question. When we build the enviroment , we run "pip install pycocotools" comment to install pycocotools package in our python site package. which one is used in this code? pycocotools2.0 in python site package or pycocotools 1.0 in coco-caption folder?

I have a repo of modified coco-caption/ for python 3. Did you try it, clone it to replace your current coco-caption/ dir. Here: https://github.com/luulinh90s/coco-caption Vào 21:54, T.4, 22 Th5, 2019 yizhiyua [email protected] đã viết: I try your training code. But return the "KeyError: 'type'" Traceback (most recent call last): File "/home/gyq/workspace/nbt3/main.py", line 215, in dataset = DataLoader(opt, split='train') File "/home/gyq/workspace/nbt3/misc/dataloader_coco.py", line 118, in init self.coco_train = COCO(det_train_path) File "tools/coco-caption/pycocotools/coco.py", line 79, in init self.createIndex() File "tools/coco-caption/pycocotools/coco.py", line 96, in createIndex if self.dataset['type'] == 'instances': KeyError: 'type' How can I Slove this problem? It seems the "instance2014.json " don't have the key "type". — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4?email_source=notifications&email_token=AHLNPF2G4UC6KKEXVXWJ65LPWU7B5A5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GVGQISQ>, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPFZRTDOPFJHWEQOKN2LPWU7B5ANCNFSM4HOUACBA .

ghost avatar May 23 '19 10:05 ghost

No

Use my modified pycocotools will work smoothly on python3

Vào 19:33, T.5, 23 Th5, 2019 yizhiyua [email protected] đã viết:

Thank you for your patient reply. I have a question. When we build the enviroment , we run "pip install pycocotools" comment to install pycocotools package in our python site package. which one is used in this code? pycocotools2.0 in python site package or pycocotools 1.0 in coco-caption folder?

I have a repo of modified coco-caption/ for python 3. Did you try it, clone it to replace your current coco-caption/ dir. Here: https://github.com/luulinh90s/coco-caption Vào 21:54, T.4, 22 Th5, 2019 yizhiyua [email protected] đã viết: … <#m_8846761804044414623_m_4939570270330131816_> I try your training code. But return the "KeyError: 'type'" Traceback (most recent call last): File "/home/gyq/workspace/nbt3/main.py", line 215, in dataset = DataLoader(opt, split='train') File "/home/gyq/workspace/nbt3/misc/dataloader_coco.py", line 118, in init self.coco_train = COCO(det_train_path) File "tools/coco-caption/pycocotools/coco.py", line 79, in init self.createIndex() File "tools/coco-caption/pycocotools/coco.py", line 96, in createIndex if self.dataset['type'] == 'instances': KeyError: 'type' How can I Slove this problem? It seems the "instance2014.json " don't have the key "type". — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4 https://github.com/luulinh90s/Neural-Baby-Talk-python3/issues/4?email_source=notifications&email_token=AHLNPF2G4UC6KKEXVXWJ65LPWU7B5A5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GVGQISQ>, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPFZRTDOPFJHWEQOKN2LPWU7B5ANCNFSM4HOUACBA .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/luulinh90s/Neural-Baby-Talk-python3/issues/4?email_source=notifications&email_token=AHLNPF6YHFBAJC5O24Q726DPWZXHBA5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWBZWYI#issuecomment-495164257, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPFZATOJJBSJ66WEZWVLPWZXHBANCNFSM4HOUACBA .

giangnguyen2412 avatar May 23 '19 11:05 giangnguyen2412

Use the one in coco caption folder

Vào 20:56, T.5, 23 Th5, 2019 Giang Nguyễn [email protected] đã viết:

No

Use my modified pycocotools will work smoothly on python3

Vào 19:33, T.5, 23 Th5, 2019 yizhiyua [email protected] đã viết:

Thank you for your patient reply. I have a question. When we build the enviroment , we run "pip install pycocotools" comment to install pycocotools package in our python site package. which one is used in this code? pycocotools2.0 in python site package or pycocotools 1.0 in coco-caption folder?

I have a repo of modified coco-caption/ for python 3. Did you try it, clone it to replace your current coco-caption/ dir. Here: https://github.com/luulinh90s/coco-caption Vào 21:54, T.4, 22 Th5, 2019 yizhiyua [email protected] đã viết: … <#m_-461681452377467024_m_8846761804044414623_m_4939570270330131816_> I try your training code. But return the "KeyError: 'type'" Traceback (most recent call last): File "/home/gyq/workspace/nbt3/main.py", line 215, in dataset = DataLoader(opt, split='train') File "/home/gyq/workspace/nbt3/misc/dataloader_coco.py", line 118, in init self.coco_train = COCO(det_train_path) File "tools/coco-caption/pycocotools/coco.py", line 79, in init self.createIndex() File "tools/coco-caption/pycocotools/coco.py", line 96, in createIndex if self.dataset['type'] == 'instances': KeyError: 'type' How can I Slove this problem? It seems the "instance2014.json " don't have the key "type". — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4 https://github.com/luulinh90s/Neural-Baby-Talk-python3/issues/4?email_source=notifications&email_token=AHLNPF2G4UC6KKEXVXWJ65LPWU7B5A5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GVGQISQ>, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPFZRTDOPFJHWEQOKN2LPWU7B5ANCNFSM4HOUACBA .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/luulinh90s/Neural-Baby-Talk-python3/issues/4?email_source=notifications&email_token=AHLNPF6YHFBAJC5O24Q726DPWZXHBA5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWBZWYI#issuecomment-495164257, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPFZATOJJBSJ66WEZWVLPWZXHBANCNFSM4HOUACBA .

giangnguyen2412 avatar May 23 '19 11:05 giangnguyen2412

Thank You. I've tried your version of coco-caption and slove the problem above. But it seems there are some problems with Dataloader. Have you ever met this problem? Traceback (most recent call last): File "main.py", line 355, in train(epoch, opt) File "main.py", line 51, in train data = data_iter.next() File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 336, in next return self._process_next_batch(batch) File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 357, in _process_next_batch raise batch.exc_type(batch.exc_msg) KeyError: 'Traceback (most recent call last):\n File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 106, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 106, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/gyq/workspace/Neural-Baby-Talk-python3/misc/dataloader_coco.py", line 272, in getitem\n cap_seq[i,k,0] = self.wtoi[caption[j]]\nKeyError: 'a'\n'

ghost avatar May 23 '19 15:05 ghost

Key error? I think the problem is from data, not dataloader

Vào 23:59, T.5, 23 Th5, 2019 yizhiyua [email protected] đã viết:

Thank You. I've tried your version of coco-caption and slove the problem above. But it seems there are some problems with Dataloader. Have you ever met this problem? Traceback (most recent call last): File "main.py", line 355, in train(epoch, opt) File "main.py", line 51, in train data = data_iter.next() File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 336, in next return self._process_next_batch(batch) File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 357, in _process_next_batch raise batch.exc_type(batch.exc_msg) KeyError: 'Traceback (most recent call last):\n File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 106, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 106, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/gyq/workspace/Neural-Baby-Talk-python3/misc/dataloader_coco.py", line 272, in getitem\n cap_seq[i,k,0] = self.wtoi[caption[j]]\nKeyError: 'a'\n'

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/luulinh90s/Neural-Baby-Talk-python3/issues/4?email_source=notifications&email_token=AHLNPF4SIWGJXZKMCZ7OXTLPW25ORA5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWCV76Q#issuecomment-495280122, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPF6NF5NKHUCJOTWC7LTPW25ORANCNFSM4HOUACBA .

giangnguyen2412 avatar May 23 '19 16:05 giangnguyen2412

Anyway,

Someone has the same problem but in the repo of the paper author Jiasen Lu. Could you pls take a look at his repo 's issues?

Vào 23:59, T.5, 23 Th5, 2019 yizhiyua [email protected] đã viết:

Thank You. I've tried your version of coco-caption and slove the problem above. But it seems there are some problems with Dataloader. Have you ever met this problem? Traceback (most recent call last): File "main.py", line 355, in train(epoch, opt) File "main.py", line 51, in train data = data_iter.next() File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 336, in next return self._process_next_batch(batch) File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 357, in _process_next_batch raise batch.exc_type(batch.exc_msg) KeyError: 'Traceback (most recent call last):\n File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 106, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/gyq/anaconda3/envs/neuralbabytalk/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 106, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/gyq/workspace/Neural-Baby-Talk-python3/misc/dataloader_coco.py", line 272, in getitem\n cap_seq[i,k,0] = self.wtoi[caption[j]]\nKeyError: 'a'\n'

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/luulinh90s/Neural-Baby-Talk-python3/issues/4?email_source=notifications&email_token=AHLNPF4SIWGJXZKMCZ7OXTLPW25ORA5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWCV76Q#issuecomment-495280122, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPF6NF5NKHUCJOTWC7LTPW25ORANCNFSM4HOUACBA .

giangnguyen2412 avatar May 23 '19 23:05 giangnguyen2412

Refer here: https://github.com/jiasenlu/NeuralBabyTalk/issues/33

giangnguyen2412 avatar May 24 '19 01:05 giangnguyen2412

Thank you for your reply. I've solved the keyerror 'type' with the new version of pycocotools in cocoapi, not using the one in coco-caption folder. I don't know if it's right, but it can work. And the second problem is caused by the dic_coco.json file. I changed the file by mistake. Thank you for your patient reply. Very appreciate. If I have other questions, hoping to get your instruction later.

ghost avatar May 24 '19 05:05 ghost

Warmly welcome :) we are all studying

Vào 13:01, T.6, 24 Th5, 2019 yizhiyua [email protected] đã viết:

Thank you for your reply. I've solved the keyerror 'type' with the new version of pycocotools in cocoapi, not using the one in coco-caption folder. I don't know if it's right, but it can work. And the second problem is caused by the dic_coco.json file. I changed the file by mistake. Thank you for your patient reply. Very appreciate. If I have other questions, hoping to get your instruction later.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/luulinh90s/Neural-Baby-Talk-python3/issues/4?email_source=notifications&email_token=AHLNPF5AEGAHMQK6MXMZR4TPW5ZC3A5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWEFCWY#issuecomment-495472987, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPFZCXZYOZKZIEWKMPL3PW5ZC3ANCNFSM4HOUACBA .

giangnguyen2412 avatar May 24 '19 05:05 giangnguyen2412

Hi, sry to bother you again. It seems there is some problem with the evaluation. Have you ever met this probelm? Traceback (most recent call last): File "/home/qyg/workspace/nbt3/main.py", line 358, in lang_stats = eval(opt) File "/home/qyg/workspace/nbt3/main.py", line 170, in eval lang_stats = utils.noc_eval(predictions, str(1), opt.val_split, opt) File "/home/qyg/workspace/nbt3/misc/utils.py", line 316, in noc_eval out = score_dcc(gt_template_novel, gt_template_train, pred, noc_object, split, cache_path) File "tools/sentence_gen_tools/coco_eval.py", line 123, in score_dcc score_dict = dcc_evaluator.get_dcc_scores() File "tools/sentence_gen_tools/coco_eval.py", line 58, in get_dcc_scores score, scores = scorer.compute_score(gts, res) File "tools/coco-caption/pycocoevalcap/bleu/bleu.py", line 23, in compute_score assert(gts.keys() == res.keys()) AssertionError The key in gts and res doesn't match

ghost avatar May 29 '19 14:05 ghost

Sorry I am busy these days. Solve it yourself and if not, reply here and I will go back soon Giang. Nguyen

On Wed, May 29, 2019 at 11:33 PM yizhiyua [email protected] wrote:

Hi, sry to bother you again. It seems there is some problem with the evaluation. Have you ever met this probelm? Traceback (most recent call last): File "/home/qyg/workspace/nbt3/main.py", line 358, in lang_stats = eval(opt) File "/home/qyg/workspace/nbt3/main.py", line 170, in eval lang_stats = utils.noc_eval(predictions, str(1), opt.val_split, opt) File "/home/qyg/workspace/nbt3/misc/utils.py", line 316, in noc_eval out = score_dcc(gt_template_novel, gt_template_train, pred, noc_object, split, cache_path) File "tools/sentence_gen_tools/coco_eval.py", line 123, in score_dcc score_dict = dcc_evaluator.get_dcc_scores() File "tools/sentence_gen_tools/coco_eval.py", line 58, in get_dcc_scores score, scores = scorer.compute_score(gts, res) File "tools/coco-caption/pycocoevalcap/bleu/bleu.py", line 23, in compute_score assert(gts.keys() == res.keys()) AssertionError The key in gts and res doesn't match

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/luulinh90s/Neural-Baby-Talk-python3/issues/4?email_source=notifications&email_token=AHLNPFZUSTFGQ6YFJUVB7QTPX2H3NA5CNFSM4HOUACBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWPQ3OA#issuecomment-496963000, or mute the thread https://github.com/notifications/unsubscribe-auth/AHLNPF5ZUZDCVQIJFUV6VG3PX2H3NANCNFSM4HOUACBA .

giangnguyen2412 avatar May 30 '19 00:05 giangnguyen2412

Hi @yizhiyua , here, in BLEU, we are computing the overlapping between label caption and predicted captions, but the keys are not match, then I think your data has a problem. Did you solve it? sorry I just came back from my final exams.

giangnguyen2412 avatar Jun 12 '19 05:06 giangnguyen2412

hi, @yizhiyua did you solve that assertion error. I have got the same problem with BLEU evaluation.

PRYUS avatar Jul 08 '19 19:07 PRYUS

when I

coco-caption/ dir. Here: https://github.com/luulinh90s/coco-caption when I use your coco-caption dir ,there still exsists type error??? image

Longcuirong avatar Jan 10 '20 06:01 Longcuirong