PPLM
PPLM copied to clipboard
Add options for more pretrained models
Hi,
I'm trying PPLM with XLNet instead of GPT-2 and encountered errors regarding the output format. It seems that the GPT-2 model has 3 output values, but the XLNet model has only 2.
/content/PPLM/run_pplm.py in generate_text_pplm(model, tokenizer, context, past, device, perturb, bow_indices, classifier, class_label, loss_type, length, stepsize, temperature, top_k, sample, num_iterations, grad_length, horizon_length, window_length, decay, gamma, gm_scale, kl_scale, verbosity_level)
594 past = model(output_so_far[:, :-1])[1]
595
--> 596 unpert_logits, unpert_past, unpert_all_hidden = model(output_so_far)
597 unpert_last_hidden = unpert_all_hidden[-1]
598
ValueError: not enough values to unpack (expected 3, got 2)
What should I try? Could you give me some hints?
My experimental notebook: https://colab.research.google.com/drive/1ywRuiFMC1NTXiIpg0xTihF8ljP0W1ECS
Thank you!
Do you know what is XLNet outputting instead of those 3 tensors?