yoyodyne
yoyodyne copied to clipboard
Hard Monotonic Transducer
(Adding on issues board for documentation, this PR will be out over the week.)
Wu and Cotterell's papers on strong alignment seem just up our alley for the library. There should be an implementation of https://aclanthology.org/P19-1148/ , particularly the monotonic cases.
Currently I have going a version that allows the following variants:
- Hard alignment. Non-monotonic.
- Hard alignment. Monotonic.
- Hard alignment. First order monotonic.
- and 3) should be good for most tasks, 1) should be available for more niche cases.
Things to add on to the PR once it's up (this makes sense to me now, will makes sense with accompanying PR).
- Original implementation stores all alignment and emission probabilities across the decoding as a cache for loss calculation. This seems unnecessary and should be a running sum.
- Ideally, the validation step should give the loss for the gold transcription. However, this requires taking into account decoding passes that produce predictions longer/shorter than the gold. Need to find a good way to penalize this while still making the loss intuitable.
- Need to double check, but I believe the prediction pass the decoder can be offloaded to our prexisting attention module.
@kylebgorman @Adamits Any additional preferences during development? I've been dancing back and forth with adding in the Ahroni and Goldberg Transducer too, just for completeness. (This and the Swissmen's transducer both supercede that one.)