WIP Accelerate enabled open source models

[thepok]

Many TODOs but already workable.

Now its easy to load big open source models.

I was able to load neox 20b and facebook 30b on my laptop with 64gb ram and 16gb vram.

[delip]

nice! im not super familiar with accerlated stuff. how generic is the get_accelerated_gpt code?

@hwchase17 Since I ran into similar issue and I am familiar with this, I left a bunch of comments to @thepok in a review.

@thepok Besides the review comments, can you add an example of how Accelarate is used (preferably not as a notebook for review purposes)? Also Accelerate is very general purpose name to use for this. In general, try leaving generic names for use in langchain core.

[thepok]

@delip Thank you for your time reviewing this. i have no interest at the moment to implement this further. Feel free to implement your suggestions your self. It could be possible to merge it in the huggingface llm loader for maximum impact.

[dev2049]

stale

[Bec-k]

ugh, so this have not been integrated?

[thepok]

nope, but i think huggingfaces transformer lib can it now by itself, and hugginfaces transformer is already in langchain :)