Jelmer
This comes mostly from me working on re-alignment. I'm moving the grand design I had in #26 to here, and make that pull request more about just supporting alignment info...
In this bit of code, we use `islice` to read `batch_size` elements from the dataset if the dataset has not yet read past its epoch (for this stage): https://github.com/hplt-project/OpusTrainer/blob/b6355ae91b7d909e161bf7ad55cce65d2184bb8f/src/opustrainer/trainer.py#L629-L635 The...
There's already a tensorboard-marian connector. We can either plug into that or write our own version of it. We have the added benefit of having direct access to marian's stdout...
Right now the trainer only attempts to write the state file when it receives a shutdown signal. Sometimes a machine just dies, and no such signal is received. It would...