petals
petals copied to clipboard
bigscience-workshop
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
[as suggested by @GreenFatGuy ] As of now, running tests locally is inconvenient as it requires spinning up servers. We cannot afford to launch servers for every test as it...
Why: - contributors who support the swarm over long time should feel that they are appreciated - one client should not be able to DDOS the entire swarm - it...
Hello, I'm trying to make the `DistributedBloomForCausalLM` work with our library [`inseq`](https://github.com/inseq-team/inseq) to extract feature attributions from BLOOM generations. However, at the moment I am facing some issues that prevent...
I do not understand how this can be stable since you are relying on donating memory and compute? I would love to use this concept in production over AWS but...
Hello, I believe the Latest bump, "Bump transformers and accelerate versions (#554)," looks to destroy Falcon support. Falcon is an essential model, given its open license use (40 and lower)....
Hi, I was wondering if anyone had recommendations of which parts of the library I should look at if I'm interested in how models are sharded, and if there's a...
Make sure that `RemotePastKeyValues` will reorder beams when the method `reorder_cache` is called. Related: #570 - [ ] Write down a code for reordering - [ ] Add Beam-Search scores...
Hi Team, I love your work, I am thinking at a very early time in the morning that we could reach a wider audience if we compiled to webassembly as...
Make possible to use up-to-date PEFT library. TODO: - [x] Fix initialization - [x] Fix inference without adapters - [x] Fix init for quantized modules - [x] Fix debug messages...
Metadata
Owner
Metadata
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading