Jay Zhou

Hi, is there an interface to specify logits processors as in vLLM? If possible, could you specify how we can customize the sampling behavior during generation?

use md5 for mac.

1

Related to facebookresearch/llama@4649acd

Hi there, this commit fixes a tiny typo in the provided pytorch text classification pipeline.

Potential Bug Report

1

# Potential Bug Hi, thanks for sharing the code. When I was trying to get the code running, I encountered the following error message: ```bash pytorch_lightning.utilities.exceptions.MisconfigurationException: ModelCheckpoint(monitor='val_acc') not found in...

`RecursionError: maximum recursion depth exceeded while calling a Python object` when inferencing with long input

6

Hi, I ran across this issue during inference ```bash Exception in ModelRpcClient: Traceback (most recent call last): File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/model_rpc.py", line 168, in exposed_step self.forward_step() File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context...