AamodThakur
AamodThakur
We are getting error after getting response from gemini. Error: "The `response.text` quick accessor only works for simple (single-`Part`) text responses. This response is not simple text.Use the `result.parts` accessor...
**Goal:** To speedup merges extraction for the Sentence Piece Adding multiprocessing for sentencepiece_extractor code to speed-up the merges extraction process. Since for 128K vocabulary more than an hour of time...
### Reproduction ```python import torch import os from datasets import Dataset, load_dataset from trl import GKDConfig, GKDTrainer from transformers import ( AutoModelForCausalLM, AutoTokenizer, ) import trl print(trl.__version__) devices="cuda:0" tokenizer =...