Jeffser
model names hard coded in instance manager will solve many issues
Pros
1 It doesn't need to check the API every time, and some crashes and basic issues can be fixed with this.
2 And the app opening will be faster.
Cons
1 You need to track all the API providers to know which models are added and constantly update the manager so users can use the latest models.
Future Idea:
I was thinking if it can be done in the future that it will fetch the models, and instead of fetching the model names everytime we open the app, it will cache the names or keep them for 72 hours and flush them upon a new API request, and so on. This fixes both sides, but I don't know how to do this.
The available models in Ollama are hardcoded since there's no API to check them.
Every other type of model with every other type of instance are checked via their respective APIs, I don't understand what you want
for suppose gemini api dont need to make aoi query when we open the app this will make the app better. but we can make it a variable like app will query for models every 72 hours and keep the modes list hardcoded so it dont have to make any querry for subsequent app opening this will make the app better i think.
def get_local_models(self) -> list:
"""Return available Gemini models"""
return [
{
'name': 'gemini-2.5-pro-preview-03-25',
'display_name': 'Gemini 2.5 Pro Preview',
'capabilities': 'Audio, images, videos, and text',
},
{
'name': 'gemini-2.0-flash',
'display_name': 'Gemini 2.0 Flash',
'capabilities': 'Audio, images, videos, and text',
},
{
'name': 'gemini-2.0-flash-lite',
'display_name': 'Gemini 2.0 Flash-Lite',
'capabilities': 'Audio, images, videos, and text',
}
]
So basically a cache so that Alpaca doesn't call the APIs constantly?
That could make sense in online instances, I don't think Ollama would need it though.
I'll work on it for the next release
So basically a cache so that Alpaca doesn't call the APIs constantly?
right this is only for the api service.
def get_local_models(self) -> list: """Return available Gemini models""" return [ { 'name': 'gemini-2.5-pro-preview-03-25', 'display_name': 'Gemini 2.5 Pro Preview', 'capabilities': 'Audio, images, videos, and text', }, { 'name': 'gemini-2.0-flash', 'display_name': 'Gemini 2.0 Flash', 'capabilities': 'Audio, images, videos, and text', }, { 'name': 'gemini-2.0-flash-lite', 'display_name': 'Gemini 2.0 Flash-Lite', 'capabilities': 'Audio, images, videos, and text', } ]
Doing it this way will put even more strain and effort onto Alpaca development, as we / Jeffry would have to keep track of every instance's available models, update the list accordingly, manage even more hard-coded stuff and would need to react very quickly once an instance removes availability of any given model (because Alpaca would still list it, yet wouldn't be able to use it.)
Doing it with a dynamically and - most importantly - automatically built cache sounds like a nice approach.
Doing it this way will put even more strain and effort onto Alpaca development, as we / Jeffry would have to keep track of every instance's available models, update the list accordingly, manage even more hard-coded stuff and would need to react very quickly once an instance removes availability of any given model (because Alpaca would still list it, yet wouldn't be able to use it.)
Some of us can do it... But my approach will be cached the data so it dont need to call api for model names also. and if the cached model name are hardcoded for a period of time subsequent app opening will be faster and less prone to crash. As i have seen many crash. Cache data will be flushed maybe in every 3days or maybe 2days or we can make it different for provide to provider like For deepseek openai gemini or other model provider and hosting done by themselves this can go as long as 10days retention of the data pulled but for other like firework togather or it can be 3 days and for open router it can be 2 days so on...
But caching ≠ hardcoding. We cannot do both, and hardcoding API responses reminds more of something like this:
def get_random_number():
# hardcoded random number, rolled with a dice and confirmed to be truly random!
return 4
this are 2 approaches one is cache the api called model names and other is filling the model names download by calling api but it will be automated not manually doing this will be a waste of time we cant do this manually.
I don't quite understand; do you mean
A) automatically querying the API, patching the response into the Alpaca builds and delivering the models and stuff right alongside Alpaca itself, which is to some extend "hard-coded", or
B) on each client, use caching logic to dynamically save the API responses to the DB and only periodically refresh from the APIs?
Hard coding is generally something you would want to avoid for stuff like that.
I don't quite understand; do you mean
A) automatically querying the API, patching the response into the Alpaca builds and delivering the models and stuff right alongside Alpaca itself, which is to some extend "hard-coded", or
B) on each client, use caching logic to dynamically save the API responses to the DB and only periodically refresh from the APIs?
Hard coding is generally something you would want to avoid for stuff like that.
B true saying hardcoding is not appropriate but still i hope i made the point