jan
jan copied to clipboard
feat: nitro hibernation/model unload after X time
Problem if a user has jan open but hasn't used it in a bit it's still hogging system RAM. since model loads are super fast nitro may want to unload the model after a period of no use.
Success Criteria new settings option to unload model after a certain sleep time or just a default sleep time.
Additional context nice to have
This should be on nitro inference plugin level
Should I post this issue in the nitro GitHub instead?
This is related to epic: #1714
Great one