jan icon indicating copy to clipboard operation
jan copied to clipboard

feat: nitro hibernation/model unload after X time

Open hwc0x01 opened this issue 1 year ago • 2 comments

Problem if a user has jan open but hasn't used it in a bit it's still hogging system RAM. since model loads are super fast nitro may want to unload the model after a period of no use.

Success Criteria new settings option to unload model after a certain sleep time or just a default sleep time.

Additional context nice to have

hwc0x01 avatar Jan 06 '24 19:01 hwc0x01

This should be on nitro inference plugin level

tikikun avatar Jan 11 '24 02:01 tikikun

Should I post this issue in the nitro GitHub instead?

hwc0x01 avatar Jan 15 '24 00:01 hwc0x01

This is related to epic: #1714

imtuyethan avatar Feb 06 '24 11:02 imtuyethan

Great one

louis-jan avatar Feb 19 '24 05:02 louis-jan