Training data issue

[jeevankumarkr]

While I try to execute the "execute.py" to train the data, following error displays, can you please help me resolve this:

Traceback (most recent call last): File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\execute.py", line 300, in train() File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\execute.py", line 114, in train enc_train, dec_train, enc_dev, dec_dev, _, _ = data_utils.prepare_custom_data(gConfig['working_directory'],gConfig['train_enc'],gConfig['train_dec'],gConfig['test_enc'],gConfig['test_dec'],gConfig['enc_vocab_size'],gConfig['dec_vocab_size']) File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\data_utils.py", line 137, in prepare_custom_data data_to_token_ids(train_enc, enc_train_ids_path, enc_vocab_path, tokenizer) File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\data_utils.py", line 112, in data_to_token_ids vocab, _ = initialize_vocabulary(vocabulary_path) File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\Scripts\easy_seq2seq\data_utils.py", line 87, in initialize_vocabulary rev_vocab.extend(f.readlines()) File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 131, in readlines s = self.readline() File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 124, in readline return compat.as_str_any(self._read_buf.ReadLineAsString()) File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\util\compat.py", line 106, in as_str_any return as_str(value) File "C:\Users\gssjekum\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\util\compat.py", line 84, in as_text return bytes_or_text.decode(encoding) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 1: invalid start byte

[joa23]

Same problem here.

[randomrandom]

same here

[absentdream]

note i had to convert all files to utf-8 and that also included the file generate (aka data/train.dec.ids20000 as data/train.enc.ids20000) so convert download data first run one and convert train.enc.ids20000 run again and convert train.dec.ids20000 ..

so i can only assume the bit that writes said files needs a tweek.. now "creating 3 layers of 256 units"... mmm so lets see what happens next (many cpu cycles later... :) )

[athuldevin]

create a folder named data at the project's main folder and place following files in it

download files from this urls manually and place on data folder

https://www.dropbox.com/s/ncfa5t950gvtaeb/test.enc?dl=0 https://www.dropbox.com/s/48ro4759jaikque/test.dec?dl=0 https://www.dropbox.com/s/gu54ngk3xpwite4/train.enc?dl=0 https://www.dropbox.com/s/g3z2msjziqocndl/train.dec?dl=0