can-ai-code

can-ai-code copied to clipboard

the-crypt-keeper

Support AQLM

1

https://github.com/Vahe1994/AQLM

the-crypt-keeper

the-crypt-keeper

Evaluate deepseek-ai/deepseek-llm-67b-base

1

the-crypt-keeper

Investigate problems with ctranslate2

7

Is it possible to investigate the problems of ctranslate2 in more detail? The library is one of the fastest and supports token streaming. Unfortunately with beam search no token streaming...

SebastianBodza

2bit llama70b with quip-sharp

1

https://github.com/Cornell-RelaxML/quip-sharp

the-crypt-keeper

Evaluate Falcon-180B with OmniQuant

1

https://github.com/OpenGVLab/OmniQuant 65GB model after quantization here, claims to run in a single A100-80GB.

the-crypt-keeper

Evaluate llama2 with low-bit-llama

1

https://github.com/GreenBitAI/low_bit_llama 2 bit quants with performance figures that are difficult to believe. Model: https://huggingface.co/GreenBitAI/LLaMA-2-70B-2bit-groupsize8

the-crypt-keeper

https://huggingface.co/BlinkDL/rwkv-4-world 15% code in the training set, very different architecture then most models tested so far

the-crypt-keeper

Model: https://huggingface.co/Salesforce/codet5p-16b this is the precursor to codegen

the-crypt-keeper