Task specific combination of the intermediate layer representations

[alberduris]

According to [Peters et al., 2018], ELMo is a task specific combination of the intermediate layer representations in the biLM.

The computation of ELMo embeddings is done by a task specific weighting of all biLM layers [1]

How is this task thing handled in the ELMoForManyLangs implementation?

To my understanding, this is a general purpose implementation that aims to generate non-task-specific contextual embeddings, so I suppose there are not task-related parameters at all. Am I right?

Thank you very much for shedding some light on this.

Alber.

[allanj]

I think they output the weights for all the layers.