hansel
hansel
Few questions: 1) how are you planning to support locally run LLMs? have an ugly hack here https://github.com/hanselke/AICodeBot/commit/ea7ccae9ef07b58c645549490447849c2e9fb76c Think thats the wrong approach tho, to run the LLMs within the...
mm think i see your problem. i think it would be best resolved by breaking up code into functions. one function per file seems like the easy prototype. Then you...
It’s think that’s prohibitively expensive tho. Costs USD$2-3 per call @32k input, and when actively using it, you are probably calling it 5-10 times an hour. That being said, the...
Yeap. Was gonna try with https://github.com/bigscience-workshop/petals to see if that works. Have work this week I need to get done tho, hopefully I’ll be able to find time next week....
cool. i didnt know that you could self-host models thru huggingfacehub. so from my research, it seems like we're gona need >20B params for it to be of any use....
hey is this gona be finally merged? looks like its failing on a macos thing, maybe the contributor doesnt have a mac? not familiar with your build scripts and i...
+1, the workaround seems clunky, gona have to write test wrappers for every function + then now test it. the jinja pipeline doesnt run on test input strings? best i've...