OpenKiwi icon indicating copy to clipboard operation
OpenKiwi copied to clipboard

Add transformer encoder fine-tuning with Adapters

Open daandouwe opened this issue 4 years ago • 0 comments

Adapters are pretty cool. Let's see if we can implement them for our Transformer encoder models by integrating with Adapter Transformers.

A particular cool extension would be AdapterFusion, which would allow us to glue together Kiwi Adapter Transformer models for various language pairs into a single multilingual Kiwi model. This idea is a QE version of the the adapters for cross-lingual transfer. Lucky for us, AdapterFusion is also implemented in Adapter Transformers.

daandouwe avatar Nov 13 '20 12:11 daandouwe