ctrl

ctrl copied to clipboard

tips and scripts related to data collection

1

Hi, Thanks for sharing your great work here! Was wondering if and when you would be releasing tips and scripts related to data collection used to pretrained CTRL. Cheers

alvinchangw

Hello! How to extract control code from text?

dedededefo

I think you should add a figure to show that.

zhaowei-wang-nlp

How to finetuning with lower memory fp16 version for p100 GPUs?

5

For finetuning with lower memory fp16 version（for fp32 version , OOMs occur. ）, How should I modify the training.py script?

xurongqiang

repeats the last word on AWS

1

When trying to run CTRL on EC2, ```Python python2 generation.py --model seqlen512_v1.ckpt/model.ckpt-740000.data-00000-of-00001 --temperature 0.2 --topk 5 ``` I get results that look like this "Wikipedia Jerry Jerry Jerry Jerry Jerry...

dataSci-rigo

I am trying to fine-tune the model but as the model is too large so it gets out of memory error. Any idea how to get over it? `tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM...

jamalabdul1

Hi ! Wondering if smaller-sized, pre-trained ctrl models, e.g., the same size as the gpt-2 model on huggingface, are available - ideal for smaller-scale experiments without large-memory GPUs :)) Thanks...

zichaow

CTRL model can not work in huggingface transformers

2

torch ver: 1.8.1+cu102 transformers ver: 4.4.2 I adopt the example codes from https://github.com/huggingface/transformers/blob/master/examples/text-generation/run_generation.py to generate text by using ctrl. here is the head part of my codes: ``` import torch...

yananchen1989

Hi, I'm running the [Colab exercise for pytorch](https://colab.research.google.com/drive/1hVveBQShDru1Mjnhe4C21uQv4A2eH1tV#scrollTo=j-XvLn5JvA5U). When I get to the last code block: `!python2 generation.py --model seqlen256_v1.ckpt/model.ckpt-413000.data-00000-of-00001` I'm getting this error: `/usr/local/lib/python2.7/dist-packages/OpenSSL/crypto.py:14: CryptographyDeprecationWarning: Python 2 is no...

yaw2014

why set "seq_length = min(args.generate_num, 256)"

1

I notice you have three pretrained models, include seqlen256_v1.ckpt and seqlen512_v1.ckpt. And you say "Only difference is the sequence length used during training. The 512 model uses double the number...

luweishuang