Dual-Learning raises NotImplementedError

[alekzieba]

...
I:0:1027-182128:[hel:par:166]:           task_name = qag                 
I:0:1027-182128:[hel:par:166]:          test_files = ['./sample_data/sample.json']
I:0:1027-182128:[hel:par:166]:         train_files = ['./sample_data/sample.json']
I:0:1027-182128:[hel:par:166]:    use_answer_masks = True                
I:0:1027-182128:[hel:par:166]:        use_coverage = True                
I:0:1027-182128:[hel:par:166]:            use_oovs = True                
I:0:1027-182128:[hel:par:166]:        weight_decay = 0                   
I:0:1027-182128:[hel:par:166]:word_embedding_files = ['./sample_data/sample.embed.txt']
I:0:1027-182128:[voc:loa: 87]:loading word embedding from ./sample_data/sample.embed.txt
I:0:1027-182128:[voc:loa:108]:building token-id map...
I:0:1027-182128:[voc:loa:130]:size of embedding 1004 x 256
I:0:1027-182128:[voc:loa:132]:size of pretrain embedding 1000 x 256
Traceback (most recent call last):
  File "app.py", line 15, in <module>
    run()
  File "app.py", line 11, in run
    getattr(__import__('api'), sys.argv[1])(args)
  File "/home/alek/Desktop/daanet-env/daanet-2/api.py", line 15, in train
    model = build_model(args)
  File "/home/alek/Desktop/daanet-env/daanet-2/utils/helper.py", line 323, in build_model
    return rccore.RCCore(args)
  File "/home/alek/Desktop/daanet-env/daanet-2/daanet/basic.py", line 24, in __init__
    super().__init__(args)
  File "/home/alek/Desktop/daanet-env/daanet-2/daanet/base.py", line 16, in __init__
    super().__init__(args)
  File "/home/alek/Desktop/daanet-env/daanet-2/base/base_model.py", line 30, in __init__
    self.data_io = dataio.DataIO(args)
  File "/home/alek/Desktop/daanet-env/daanet-2/dataio_utils/full_load_io.py", line 14, in __init__
    ModeKeys.TRAIN: self.load_data(self.args.train_files, ModeKeys.TRAIN),
  File "/home/alek/Desktop/daanet-env/daanet-2/base/base_io.py", line 23, in load_data
    raise NotImplementedError
NotImplementedError

Originally posted by @alekzieba in https://github.com/hanxiao/daanet/issues/2#issuecomment-433660167

[zcc616]

I have same issue with you, haven't you solve this problem ?

[alekzieba]

@zcc616 Unfortunately, I could not solve this issue, but I noticed that the line that is throwing the error is invoked due to dataio_utils/full_load_io.py calling the following in base/base_io.py:

def load_data(self, file_paths: List[str], mode: ModeKeys):
    raise NotImplementedError

So I'm not sure if this was intentional by the authors (i.e. if we are supposed to implement this ourselves, or if there is something else that I missed); in the end, I could not get past this step.

[MarvinLong]

@alekzieba It raises NotImplementedError, do you implement it ? I think some methods need to be implemented by ourselves.