idea: Local Provider Discoverability with Zeroconfiguration

[jperrello]

Problem Statement

Users of Jan want to start chatting immediately with absolutely no setup on their end.

Feature Idea

Implement ZeroconfAI! https://github.com/jperrello/Zeroconf-AI Technical details: Jan does an mDNS lookup for _zeroconfai._tcp.local. and the results describe OpenAI compatible endpoints for any providers that announce themselves on the local area network.

Disclaimer: I am the main developer for zeroconfAI and I would love to have Jan be the first app to use this so I can start building a much larger ecosystem. This is not a commercial effort, it is my Master's Project in Baskin Engineering at UCSC.

We added support for ZeroConf AI to Jan in a roundabout way. We created a Python script that listens for all local service announcements and runs a local proxy server that is OpenAI compatible. Ideally, Jan would be able to listen for these announcements itself. But after that, the way the servers are communicated with is exactly the same as any other OpenAI compatible server. No API keys are needed anywhere.