[Discussion] information leakage while intaking the conversition?

[james20141606]

Discussion

Hi, I have one concern about the model taking in the conversation. One conversation might have multiple questions and answers and it seems (from my understanding) the model is tokenizing and consuming the QAs in one conversation simultaneously. I wonder if my understanding is True, and if yes, could it influence the model? If some answers are based on the previous question, could that lead to some potential information leakage?

[mu-cai]

Yeah! This is a good question! A perfect conversational data should aviod such lekage!