Pytorch-seq2seq-Beam-Search
Pytorch-seq2seq-Beam-Search copied to clipboard
PyTorch implementation for Seq2Seq model with attention and Greedy Search / Beam Search for neural machine translation
Pytorch-seq2seq-Beam-Search
Seq2Seq model with attention and Greedy Search / Beam Search for neural machine translation in PyTorch.
This implementation focuses on the following features:
- Modular structure to be used in other projects
- Minimal code for readability
- Full utilization of batches and GPU.
- Decoding Method Greedy Search
- Decoding Method Beam Search
This implementation relies on torchtext to minimize dataset management and preprocessing parts.
Seq2Seq Model description
The main structure of seq2seq is adopt in seq2seq
- Encoder: Bidirectional GRU
- Decoder: GRU with Attention Mechanism
- Attention
Decoding Method
- Greedy Search
- Beam Search
Requirements
- CUDA
- Python 3.6
- PyTorch 1.4
- torchtext
- Spacy
- numpy
- Visdom (optional)
download tokenizers by doing so:
python -m spacy download de
python -m spacy download en
TODO
- add logging
- change to support gpu and cpu, currently is gpu based code implementation
- Do Length normalization experiment on Beam-Search And Compare
References
Based on the following implementations