MatchingNetworks
MatchingNetworks copied to clipboard
AntreasAntoniou
Data loader issue
hey,
I'm using my own dataset which follows the folder structure as given in the readme. I have 89 classes and want to split 64 10 15 as the train validation and evaluation split . Each class is balanced and has 20 samples. I made the following changes: data = dataset.FolderDatasetLoader(num_of_gpus=1, batch_size=args.batch_size, image_height=200, image_width=300, image_channels=3, train_val_test_split=(64/89, 10/89, 15/89), samples_per_iter=1, num_workers=4, data_path="datasets/city_names", name="city_names", indexes_of_folders_indicating_class=-2, reset_stored_filepaths=False, num_samples_per_class=args.samples_per_class, num_classes_per_set=args.classes_per_set, label_as_int=False) and getting the following error Mapped data paths can't be found, remapping paths.. Get images from /home/user/MatchingNetworks/datasets/city_names Traceback (most recent call last): File "/home/cvprisi19/MatchingNetworks/data.py", line 98, in load_datapaths data_image_paths = self.load_dict(data_path_file) File "/home/cvprisi19/MatchingNetworks/data.py", line 115, in load_dict with open(name, 'rb') as f: FileNotFoundError: [Errno 2] No such file or directory: 'datasets/city_names.pkl'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train_one_shot_learning_matching_network.py", line 20, in
even though the path contains the dataset and the images ...the pickle file itself is not being generated.
I see. Let's diagnose this together.
- Do you have write access to the folder? If yes, move to question 2.
- Search for .pkl files in your whole framework directory and let me know if you can find the currently missing pkl file.
- Yes I have read/write access to the folder.
- The pkl files are not being generated that's the main issue. I think the program is not able to locate the dataset . I'm guessing it has something to do with this argument below indexes_of_folders_indicating_class=-2, does this seem right for the dataset structure that I have?
I see. Let's diagnose this together.
- Do you have write access to the folder? If yes, move to question 2.
- Search for .pkl files in your whole framework directory and let me know if you can find the currently missing pkl file.
, I kind of solved the data loader issue, now i'm able to togenerate the pkl files and load data but after the training starts, I'm getting this error,
0%| | 0/200 [00:00<?, ?it/s]
Traceback (most recent call last): | 0/1000 [00:00<?, ?it/s]
File "train_one_shot_learning_matching_network.py", line 69, in
any insight would be useful
I also encounter the data loader issues, no PKL file is generated while running the 'train_one_shot_learning_matching_network' as instructured.
Hello everyone,
This implementation is now quite old. Expect a full update both of the code and the data providers within the next month.
On Sat, 4 Apr 2020 at 08:49, Dennis-Chu [email protected] wrote:
Can any one share the solution to make this source work?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/AntreasAntoniou/MatchingNetworks/issues/19#issuecomment-608991640, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACSK4NTRWF7RB2S45ZGTDC3RK3RA3ANCNFSM4H6HQXBA .
I see. Let's diagnose this together.
- Do you have write access to the folder? If yes, move to question 2.
- Search for .pkl files in your whole framework directory and let me know if you can find the currently missing pkl file.
, I kind of solved the data loader issue, now i'm able to togenerate the pkl files and load data but after the training starts, I'm getting this error,
0%| | 0/200 [00:00<?, ?it/s] Traceback (most recent call last): | 0/1000 [00:00<?, ?it/s] File "train_one_shot_learning_matching_network.py", line 69, in sess=sess) File "/home/cvprisi19/MatchingNetworks/experiment_builder.py", line 82, in run_training_epoch self.learning_rate: self.current_learning_rate}) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 929, in run run_metadata_ptr) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1128, in _run str(subfeed_t.get_shape()))) ValueError: Cannot feed value of shape (1, 32, 300, 200, 3) for Tensor 'support_set_images:0', which has shape '(1, 32, 5, 1, 200, 300, 3)'
any insight would be useful
请问你是如何解决你的第一个问题，也就是 data loader issue,
I also encounter the data loader issues, no PKL file is generated while running the 'train_one_shot_learning_matching_network' as instructured.
你好，请问你解决了这个问题了吗，无法生成pkl文件。。
I see. Let's diagnose this together.
- Do you have write access to the folder? If yes, move to question 2.
- Search for .pkl files in your whole framework directory and let me know if you can find the currently missing pkl file.
, I kind of solved the data loader issue, now i'm able to togenerate the pkl files and load data but after the training starts, I'm getting this error,
0%| | 0/200 [00:00<?, ?it/s] Traceback (most recent call last): | 0/1000 [00:00<?, ?it/s] File "train_one_shot_learning_matching_network.py", line 69, in sess=sess) File "/home/cvprisi19/MatchingNetworks/experiment_builder.py", line 82, in run_training_epoch self.learning_rate: self.current_learning_rate}) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 929, in run run_metadata_ptr) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1128, in _run str(subfeed_t.get_shape()))) ValueError: Cannot feed value of shape (1, 32, 300, 200, 3) for Tensor 'support_set_images:0', which has shape '(1, 32, 5, 1, 200, 300, 3)'
any insight would be useful
Would you like to share your method to solve this data loader problem? I ran the code successfully on the Linux based server, but failed on my personal computer, which is based on Windows 10.
