THUMT icon indicating copy to clipboard operation
THUMT copied to clipboard

Do you have an instruction manual for the pytorch version?

Open Felixgithub2017 opened this issue 5 years ago • 2 comments

Some commands for additional parameters are not working in the pytorch version, so, do you have an pytorch-oriented manual?

For example:

--parameters=batch_size=15000,device_list=[0,1],update_cycle=2,train_steps=2000000,keep_checkpoint_max=5,shared_embedding_and_softmax_weights=True,shared_source_target_embedding=True

raise ValueError("Could not parse hparam %s in %s" % (name, values)) ValueError: Could not parse hparam shared_embedding_and_softmax_weights in batch_size=15000,device_list=[0,1],update_cycle=2,train_steps=2000000,keep_checkpoint_max=5,shared_embedding_and_softmax_weights=True,shared_source_target_embedding=True

Felixgithub2017 avatar Jan 03 '20 06:01 Felixgithub2017

image The initial loss is inf, and turned to normal after around 200 steps

Felixgithub2017 avatar Jan 03 '20 07:01 Felixgithub2017

In the above example, you should set shared_embedding_and_softmax=true instead of shared_embedding_and_softmax=True. The document of PyTorch implementation will be uploaded soon. We have tested our implementation on several datasets, but we do not observe inf loss problem.

Playinf avatar Jan 06 '20 03:01 Playinf