sockeye icon indicating copy to clipboard operation
sockeye copied to clipboard

Add alignment matrix learning

Open iPRET opened this issue 9 months ago • 3 comments

Added alignment matrix learning to training. See issue #1105 for more details. Added command line arguments --alignment-matrix for specifying a file to read alignments from. --alignment-matrix-weight for specifying loss coefficient for alignment matrix cross entropy. --attention-alignment-layer for specifying layer of decoder in which attention alignment will happen. --align-attentions for telling model to learn alignments when that's impossible to infer from other command line arguments. --shift-alignments for telling data preparation to shift alignments one target token forward, and translation one target token backward.

Report on performance impact: Sockeye_Alignment_Matrix_Report-6.pdf

Pull Request Checklist

  • [x] Changes are complete (if posting work-in-progress code, prefix your pull request title with '[WIP]' until you can check this box.
  • [x] Unit tests pass (pytest)
  • [x] Were system tests modified? If so did you run these at least 5 times to account for the variation across runs?
  • [x] System tests pass (pytest test/system)
  • [ ] Passed code style checking (./style-check.sh)
  • [x] You have considered writing a test
  • [x] Updated major/minor version in sockeye/__init__.py. Major version bump if this is a backwards incompatible change.
  • [x] Updated CHANGELOG.md

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

iPRET avatar May 15 '24 13:05 iPRET