text-to-sql icon indicating copy to clipboard operation
text-to-sql copied to clipboard

Published 20 hours ago • verified publisher icon BrettlyCD

Using local Models

Open zzadiues opened this issue 2 years ago • 1 comments

Hi @BrettlyCD , thanks for the really instructive repo. I am having a hard time getting a local model to run this application. Have you tried a version that utilizes a local model ?

zzadiues avatar Oct 16 '23 22:10 zzadiues

Hi, I'm sorry for just now seeing this!

I've done a little bit of testing with some other projects on local models. My laptop doesn't have a ton of processing power, so I didn't have a ton of luck.

I'm finding the model restriction to really be the challenge here - using HuggingFace you are limited to the amount of input tokens. Using a local model you are restricted to just not having the best model due to size.

I have been prioritizing some other projects recently, but would love to get back into this and do some more exploring. Thank you for the question!

BrettlyCD avatar Nov 16 '23 04:11 BrettlyCD