Use of other Encode/Decoder Models

[rdmerillat]

Hello, I've been using Unlimiformer as a comparison with current standard methods of summarization and was wondering if there was anything in particular that would be needed in order to convert say a Pegasus model into Unlimiformer as it should work with "All Encoder/Decoder" models. I see several lines commented out in unlimiformer.py (here) for AutoModelForSeq2Seq, however I currently dont see a direct way this has been implemented yet.

As Pegasus is BART based, I set up a new model conveter PegasusForConditionalGeneration: UnlimiformerPegasus, and started a new unlimiformer class for it:

class UnlimiformerPegasus(UnlimiformerBART):
    def __init__(self, model: PegasusModel, *args, **kwargs):
        super().__init__(model, *args, **kwargs)

However, I was wondering if you or anyone else had found additional tweeking that was needed to fully convert say a pegasus model.

And I guess more generally, what is the procedure that you use when setting up your own new unlimiformer converted models as I was unable to simply glean what was necessary to assure "consistent" performance and or results.

Thanks!