How to pre-train some parameters of deep prompt？

[daisy-disc]

How to pre-train some parameters of deep prompt？ Pre-train deep prompts One is to pre-train deep prompts with a vanilla PLM. Later we initialize a DPT-based retriever using the pre-trained deep prompts and the vanilla PLM. However, experiments in Sec. 4.4 show that it suffers from catastrophic forgetting and exhibits no superior performance to randomly initialized prompts. how to get the "pre-trained deep prompts"?

[tangzhy]

Hi, for pre-training of deep prompts, we just employ a prompted dual-encoder to perform the RIP task, during which the PLM is fixed as a vanilla PLM while prompts is being updated.