AlbertForNextSentencePredicition

[DeepakDhanasekar]

Hey,

I was going through the code and found out that you had used NSP on pytorch Albert implementation. Doesn't Albert use Sentence Order Predicition??

[lonePatient]

@DeepakDhana SOP code:

                if random.random() < 0.5:  # 交换一下tokens_a和tokens_b
                    is_random_next = True
                    temp = tokens_a
                    tokens_a = tokens_b
                    tokens_b = temp
                else:
                    is_random_next = False

[DeepakDhanasekar]

I have seem this code in both prepare_lm_data_mask.py and prepare_lm_data_ngram.py. What are the difference between them?