l2p
l2p copied to clipboard
google-research
Method issue
Hi,
I am trying to do some experiments using l2p and there's something a little strange about l2p.
According to the contents of the paper and my understanding, the prompts in the prompt pool should be selected evenly for each task. However, when I look at the prompt index and the Tensorboard histogram, it seems that only a few prompts are learned. Only the 3rd, 6th, 7th, and 9th prompts are used in the official code of the reproduce result.
Did I misunderstand? or was the code implemented incorrectly?
I look forward to hearing from you.
Best. Jaeho Lee.
The recreated code for text classification (in NLP) has same question. I think the reason is:
First, we select a group of prompts (top k) its initialized key is most close to the querys of the first task. Then, optimize that prompts and corresponding keys. When training the second task, we select same prompts to the first one because of the keys has been pull to the centre of all task by optimization.
I think the initial method is crucial for this quesiton, but get same result even I tried all combination method mentioned in the code.
May I miss any constraint？
