Example of multi-task

[NtaylorOX]

Hi,

I saw on the documentation page there is a page for multi-task training: https://opendelta.readthedocs.io/en/latest/notes/pluginunplug.html.

However I think it is not entirely clear how this modelling approach would work in practice?

Is there any examples for using OpenDelta for multi-task with the training code etc?

Thanks in advance

Best,

Niall

[NtaylorOX]

Sorry - just found this; https://colab.research.google.com/drive/1uAhgAdc8Qr42UKYDlgUv0f7W1-gAFwGo?usp=sharing

I will see how this works and once satisfied will close issue

[NtaylorOX]

Having looked through the provided notebook - https://colab.research.google.com/drive/1uAhgAdc8Qr42UKYDlgUv0f7W1-gAFwGo?usp=sharing, it appears to show how you can load in different delta models as "serving multi-tasks" but it doesn't show how you can train multiple-tasks at the same time efficiently?

This "serving multi-task" just seems to show how to load in 3 different fine-tuned models and act on the same input sequence - is it possible with OpenDelta to train using one back-bone model with 3 different deltas? Each using its own loss sort of thing?

Any help would be much appreciated

[telxt]

Sorry, the current OpenDelta can't support training with 3 different deltas using one back-bone model. Sorry for not being able to help you, we will conduct further research and try our best to make improvements so that Open Delta can support multi-task training.