generative-ai-python
generative-ai-python copied to clipboard
Add min_p #680
Generates text one token at a time. At each step, it requests many (num_samples) candidates for the next single token. Estimates the probability of each potential next token based on its frequency in the candidates. Applies the min_p filtering logic using these estimated probabilities.