While I try to execute the "execute.py" to train the data, following error displays, can you please help me resolve this:
Traceback (most recent call last):
File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\execute.py", line 300, in
train()
File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\execute.py", line 114, in train
enc_train, dec_train, enc_dev, dec_dev, _, _ = data_utils.prepare_custom_data(gConfig['working_directory'],gConfig['train_enc'],gConfig['train_dec'],gConfig['test_enc'],gConfig['test_dec'],gConfig['enc_vocab_size'],gConfig['dec_vocab_size'])
File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\data_utils.py", line 137, in prepare_custom_data
data_to_token_ids(train_enc, enc_train_ids_path, enc_vocab_path, tokenizer)
File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\data_utils.py", line 112, in data_to_token_ids
vocab, _ = initialize_vocabulary(vocabulary_path)
File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\data_utils.py", line 87, in initialize_vocabulary
rev_vocab.extend(f.readlines())
File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 131, in readlines
s = self.readline()
File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 124, in readline
return compat.as_str_any(self._read_buf.ReadLineAsString())
File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\util\compat.py", line 106, in as_str_any
return as_str(value)
File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\util\compat.py", line 84, in as_text
return bytes_or_text.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 1: invalid start byte
note i had to convert all files to utf-8 and that also included the file generate (aka data/train.dec.ids20000 as data/train.enc.ids20000) so convert download data first run one and convert train.enc.ids20000 run again and convert train.dec.ids20000 ..
so i can only assume the bit that writes said files needs a tweek.. now "creating 3 layers of 256 units"... mmm so lets see what happens next (many cpu cycles later... :) )
create a folder named data at the project's main folder and place following files in it
download files from this urls manually and place on data folder
https://www.dropbox.com/s/ncfa5t950gvtaeb/test.enc?dl=0
https://www.dropbox.com/s/48ro4759jaikque/test.dec?dl=0
https://www.dropbox.com/s/gu54ngk3xpwite4/train.enc?dl=0
https://www.dropbox.com/s/g3z2msjziqocndl/train.dec?dl=0
suriyadeepan
