Sam Shleifer

logs/keyfreq_...txt

9

My key freq log file has timestamps, but each timestamp has a zero associated with it. When I load the webpage, there is no number next to total number of...

Streaming

1

I have a streaming endpoint. When I turn on flask-compress it stops streaming. Is there a workaround? ``` app = Flask(__name__) app.config["COMPRESS_REGISTER"] = False # disable default compression of all...

If you copy checkpoints from HOME to gcs they can get deleted

3

because of this line https://github.com/google/orbax/blob/c7a7fd48ff094ac4167d3fb1c10f8b6c8de32b3a/checkpoint/orbax/checkpoint/utils.py#L669 They don't have success file but are in GCS so orbax thinks its tmp and cleans it up. I would suggest always or never saving...

Best server cmd for mistralai/Mistral-7B-v0.1

2

``` export MODEL=mistralai/Mistral-7B-v0.1 python3 -m vllm.entrypoints.openai.api_server --model $MODEL \ --tensor-parallel-size=1 \ --enable-prefix-caching --max-model-len=4096 --trust-remote-code | tee server_mistral.log & ``` raises `NotImplementedError: Sliding window is not allowed with prefix caching enabled!`...

Request: Force the first set of evaluated parameters/ save info to trials

2

I know the best params for my model but hyperopt over a parameter space centered around my best params has thus far failed to discover params with a better objective...

Splitting tests with awareness of skips

1

I have two settings for CI a) run all the tests on GPU b) run a subset of the tests on CPU the subset is because many tests are decorated...

rm followed by exists returns true

1

Why does fs.exists() sometimes return True even after deleting directories using fsspec/s3fs? Is there a reliable way to confirm directory deletion on Cloudflare R2 in python? If I `rclone lsf...