John Vandivier
John Vandivier
fine tune dolly v2 for $30 https://www.tiktok.com/@rajistics/video/7222430618347490602
few shot or in-context learning is considered newer than fine-tuning (but is it more performant?) https://www.tiktok.com/@rajistics/video/7226905183601708331 `Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning` https://openreview.net/forum?id=rBCvMG-JsPd but what about a...
MPT-7B-StoryWriter-65k+ literally made to write books ("ALiBi, MPT-7B-StoryWriter-65k+ can extrapolate even beyond 65k tokens, and we have demonstrated generations as long as 84k tokens on a single node of A100-80GB...
ChatGLM6B
https://github.com/oobabooga/text-generation-webui/blob/main/models/config.yaml supported oobabooga models
https://www.paperspace.com/pricing
[Geforce RTX 2060](https://www.nvidia.com/en-us/geforce/graphics-cards/rtx-2060/) or better to run https://github.com/openai/triton (optimized thingy under MPT)
https://github.com/cocktailpeanut/dalai little lighter weight than oobabooga (maybe?)
web gpu acceleration https://github.com/mlc-ai/web-llm 4.8276 tokens/sec for my Nvidia geforce GTX 960, ~5.5 CUDA Compute ability, 4 gb GPU dedicated RAM
TODO: cloud dev with langchain try paperspace + A100 can i use IPUs? (paperspace)