Chris Taylor

The TonkineseNAT code needs to be updated with iOS-specific APIs to get the gateway address. There are also generally OS specific things to port around the codebase. There should be...

The TonkineseNAT code needs to be updated with Android-specific APIs to get the gateway address. There are also generally OS specific things to port around the codebase. There should be...

Copa congestion control includes three improvements over the current strategy: (1) It has a velocity term. (2) It has a TCP-competitive mode. (3) It has better inter-flow fairness. The one...

Examples: https://gist.github.com/kajott/6771de6dfca0255039103e9c4c26e034 https://github.com/catid/gcif/blob/master/decoder/Filters.cpp etc Maybe something will be better and about as fast?

Add multi-gpu support, model cache, fix README and deprecated containers

8

This adds several improvements: [x] Support for consumer GPUs for hosting [x] Local model cache for docker container to avoid re-downloading the models on restart [x] Fixes for deprecated container...

LLaMA-3 issues when used with vLLM

5

I tried these two quantization approaches: ``` model_path = '/home/catid/models/Meta-Llama-3-70B-Instruct' quant_path = 'cat-llama-3-70b-q128-w4-gemvfast' quant_config = { "zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "gemv_fast" } ``` ``` model_path = '/home/catid/models/Meta-Llama-3-70B-Instruct'...

To save you $$ on runpods, here's what I found for this model: ``` python ./compute_block_similarity/layer_similarity.py \ --model_path "Meta-Llama-3-70B-Instruct" \ --dataset "arcee-ai/sec-data-mini" \ --dataset_column "text" \ --batch_size 1 \ --max_length...

I opened a mojo ticket here: https://github.com/modularml/mojo/issues/1330