cosmoquester
Would it be possible to got a quick sentence infilling example?
First, thanks for the pretraining script!
I've only really worked in PyTorch, so far, so I'm really not familiar with how things are done in TF. Could you provide an example of how how to get a basic sentence infilling example working, or maybe point me to some sample code?
Specifically, I can't seem to load a trained checkpoint (i.e., from output/models) using TFBartForConditionalGeneration.from_pretrained().
Ah, geez... Just hadn't pointed to the config correctly. The model is loading now, sorry.
However, trying to run it with
tokenizer = tokenizers.SentencePieceBPETokenizer("./sp_model/merged_bpe.model")
model = TFBartForConditionalGeneration.from_pretrained(
"./output/models/model-10epoch-1.8778loss_0.5293acc.ckpt.data-00000-of-00001", config="./configs/mini.json"
)
input_ids = tokenizer([TXT], return_tensors="tf")["input_ids"]
logits = model(input_ids).logits
probs = tf.nn.softmax(logits[0])
is throwing an error about the GPU:
F ./tensorflow/core/kernels/random_op_gpu.h:246] Non-OK-status: GpuLaunchKernel(FillPhiloxRandomKernelLaunch<Distribution>, num_blocks, block_size, 0, d.stream(), key, counter, gen, data, size, dist) status: INTERNAL: invalid configuration argument
I'm running an RTX-2070, which is the gpu I trained on. I did notice something a little weird, in that it shows:
Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3996 MB memory: -> device: 0, name: NVIDIA GeForce RTX 2070, pci bus id: 0000:01:00.0, compute capability: 7.5
For some reason, nvidia-smi is reporting 2188 mb used, but I'm not sure why...
