mmf
mmf copied to clipboard
facebookresearch
TextVQA
❓ Questions and Help
Hello I am trying to use MMF to predict answers on images. I'd like to implement my own dataset, I tried to do that using the tutorial of adding dataset in the documentation but I always end up with something unclear. Is there any notebook, or example code to get me started? Using M4C model in inference ?
Thanks.
This branch contains an example notebook that shows how to add a dataset : https://github.com/facebookresearch/mmf/blob/notebooks/notebooks/kdd_tutorial.ipynb
Let us know if that helps or you need more information.
Hello,
Thank you very much for your responsE. Appreciate it.
I'd like to know how can we generate the ocr tokens from an image ? Using OCR tesseract API is the only way I can think of.
Regards.
De : Vedanuj Goswami @.> Envoyé : lundi 2 août 2021 17:57 À : facebookresearch/mmf @.> Cc : BENELHAJ Yasmine @.>; Author @.> Objet : Re: [facebookresearch/mmf] TextVQA (#1037)
This branch contains an example notebook that shows how to add a dataset : https://github.com/facebookresearch/mmf/blob/notebooks/notebooks/kdd_tutorial.ipynb
Let us know if that helps or you need more information.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/facebookresearch/mmf/issues/1037#issuecomment-891139441, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALYT7VROEYWPVKBXWQ52GZDT2257TANCNFSM5BKAODTQ.
@yasminabelhadj You can try out Google's Cloud Vision API for OCR text extraction.
Hello,
I am trying to execute the KDD notebook but I always run into this error:
AssertionError: OKVQA requires 2 paths; one to questions and one to annotations
either by using google colab or my own jupyter notebook.
Can you please guide me into correcting it? When I show the path in load_annotation_db it is set to : "
['/home/yasmine/.cache/torch/mmf/data/datasets/okvqa/defaults/annotations/annotations/imdb_train1.npy']
" When I entrer the directory "/home/yasmine/. cache/torch/mmf/data/" it is empty.
Thank you.
De : BENELHAJ Yasmine @.> Envoyé : jeudi 5 août 2021 09:31 À : facebookresearch/mmf @.>; facebookresearch/mmf @.> Cc : Author @.> Objet : RE: [facebookresearch/mmf] TextVQA (#1037)
Hello,
Thank you very much for your responsE. Appreciate it.
I'd like to know how can we generate the ocr tokens from an image ? Using OCR tesseract API is the only way I can think of.
Regards.
De : Vedanuj Goswami @.> Envoyé : lundi 2 août 2021 17:57 À : facebookresearch/mmf @.> Cc : BENELHAJ Yasmine @.>; Author @.> Objet : Re: [facebookresearch/mmf] TextVQA (#1037)
This branch contains an example notebook that shows how to add a dataset : https://github.com/facebookresearch/mmf/blob/notebooks/notebooks/kdd_tutorial.ipynb
Let us know if that helps or you need more information.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/facebookresearch/mmf/issues/1037#issuecomment-891139441, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALYT7VROEYWPVKBXWQ52GZDT2257TANCNFSM5BKAODTQ.
This branch contains an example notebook that shows how to add a dataset : https://github.com/facebookresearch/mmf/blob/notebooks/notebooks/kdd_tutorial.ipynb
Let us know if that helps or you need more information.
I am trying to run this notebook. But I meet an error:
AssertionError Traceback (most recent call last)
<ipython-input-18-353bc29e5b82> in <module>()
4 # Init configuration to register resolvers
5 Configuration()
----> 6 dataset = build_dataset("okvqa_colab", dataset_type="val")
7 import matplotlib.pyplot as plt
8 plt.rcParams["figure.figsize"] = (20, 20)
7 frames
/content/mmf/mmf/utils/build.py in build_dataset(dataset_key, config, dataset_type)
197
198 datamodule_instance.build_dataset(config)
--> 199 dataset = datamodule_instance.load_dataset(config, dataset_type)
200 if hasattr(datamodule_instance, "update_registry_for_model"):
201 datamodule_instance.update_registry_for_model(config)
/content/mmf/mmf/datasets/base_dataset_builder.py in load_dataset(self, config, dataset_type, *args, **kwargs)
162 DO NOT OVERRIDE in child class. Instead override ``load``.
163 """
--> 164 dataset = self.load(config, dataset_type, *args, **kwargs)
165 if dataset is not None and hasattr(dataset, "init_processors"):
166 # Checking for init_processors allows us to load some datasets
/content/mmf/mmf/datasets/mmf_dataset_builder.py in load(self, config, dataset_type, *args, **kwargs)
137 for imdb_idx in range(len(annotations)):
138 dataset_class = self.dataset_class
--> 139 dataset = dataset_class(config, dataset_type, imdb_idx)
140 datasets.append(dataset)
141
<ipython-input-10-903272dd23fe> in __init__(self, config, dataset_type, index, *args, **kwargs)
12 # The super call will build annotation database, image database and
13 # feature database based on config passed.
---> 14 super().__init__("okvqa_colab", config, dataset_type, index, *args, **kwargs)
15
16 def build_annotation_db(self):
/content/mmf/mmf/datasets/mmf_dataset.py in __init__(self, dataset_name, config, dataset_type, index, annotation_database, *args, **kwargs)
31 self._index = index
32 self.annotation_database = annotation_database
---> 33 self.annotation_db = self.build_annotation_db()
34 self._use_images = self.config.get("use_images", False)
35 if self._use_images:
<ipython-input-10-903272dd23fe> in build_annotation_db(self)
26 )
27
---> 28 return OKVQAAnnotationDatabase(self.config, annotation_path)
29
30 def get_image_path(self, image_id: str):
/content/mmf/mmf/datasets/databases/annotation_database.py in __init__(self, config, path, *args, **kwargs)
21 self.start_idx = 0
22 path = get_absolute_path(path)
---> 23 self.load_annotation_db(path)
24
25 def load_annotation_db(self, path):
<ipython-input-9-99236fd6e366> in load_annotation_db(self, path)
11 path = path.split(",")
12 assert len(path) == 2, (
---> 13 "OKVQA requires 2 paths; one to questions and one to annotations"
14 )
15
AssertionError: OKVQA requires 2 paths; one to questions and one to annotations
Can you do me a favor? Thanks very much
This branch contains an example notebook that shows how to add a dataset : https://github.com/facebookresearch/mmf/blob/notebooks/notebooks/kdd_tutorial.ipynb
Let us know if that helps or you need more information.
Why you need to git checkout ?
!git checkout 95e89ee
Hi, I'm having the same issue. Have you solved it? Can you tell me how?
This branch contains an example notebook that shows how to add a dataset : https://github.com/facebookresearch/mmf/blob/notebooks/notebooks/kdd_tutorial.ipynb Let us know if that helps or you need more information.
I am trying to run this notebook. But I meet an error:
AssertionError Traceback (most recent call last) <ipython-input-18-353bc29e5b82> in <module>() 4 # Init configuration to register resolvers 5 Configuration() ----> 6 dataset = build_dataset("okvqa_colab", dataset_type="val") 7 import matplotlib.pyplot as plt 8 plt.rcParams["figure.figsize"] = (20, 20) 7 frames /content/mmf/mmf/utils/build.py in build_dataset(dataset_key, config, dataset_type) 197 198 datamodule_instance.build_dataset(config) --> 199 dataset = datamodule_instance.load_dataset(config, dataset_type) 200 if hasattr(datamodule_instance, "update_registry_for_model"): 201 datamodule_instance.update_registry_for_model(config) /content/mmf/mmf/datasets/base_dataset_builder.py in load_dataset(self, config, dataset_type, *args, **kwargs) 162 DO NOT OVERRIDE in child class. Instead override ``load``. 163 """ --> 164 dataset = self.load(config, dataset_type, *args, **kwargs) 165 if dataset is not None and hasattr(dataset, "init_processors"): 166 # Checking for init_processors allows us to load some datasets /content/mmf/mmf/datasets/mmf_dataset_builder.py in load(self, config, dataset_type, *args, **kwargs) 137 for imdb_idx in range(len(annotations)): 138 dataset_class = self.dataset_class --> 139 dataset = dataset_class(config, dataset_type, imdb_idx) 140 datasets.append(dataset) 141 <ipython-input-10-903272dd23fe> in __init__(self, config, dataset_type, index, *args, **kwargs) 12 # The super call will build annotation database, image database and 13 # feature database based on config passed. ---> 14 super().__init__("okvqa_colab", config, dataset_type, index, *args, **kwargs) 15 16 def build_annotation_db(self): /content/mmf/mmf/datasets/mmf_dataset.py in __init__(self, dataset_name, config, dataset_type, index, annotation_database, *args, **kwargs) 31 self._index = index 32 self.annotation_database = annotation_database ---> 33 self.annotation_db = self.build_annotation_db() 34 self._use_images = self.config.get("use_images", False) 35 if self._use_images: <ipython-input-10-903272dd23fe> in build_annotation_db(self) 26 ) 27 ---> 28 return OKVQAAnnotationDatabase(self.config, annotation_path) 29 30 def get_image_path(self, image_id: str): /content/mmf/mmf/datasets/databases/annotation_database.py in __init__(self, config, path, *args, **kwargs) 21 self.start_idx = 0 22 path = get_absolute_path(path) ---> 23 self.load_annotation_db(path) 24 25 def load_annotation_db(self, path): <ipython-input-9-99236fd6e366> in load_annotation_db(self, path) 11 path = path.split(",") 12 assert len(path) == 2, ( ---> 13 "OKVQA requires 2 paths; one to questions and one to annotations" 14 ) 15 AssertionError: OKVQA requires 2 paths; one to questions and one to annotationsCan you do me a favor? Thanks very much
