Avoid user requirement of providing their keys while controlling API costs

[syltruong]

Issue

Having to set up API keys (currently OpenAI, Twitter, Reddit) can be a barrier to acquiring users.

If we were to put the Media Agent in production, the experience needs to be as seamless as possible.

• Reddit: free read-only service account API keys can be setup
• Twitter: API costs
• OpenAI pay as you go pricing

Are there alternatives/strategies in order to minimise/control cost?

[ahmedbesbes]

You can consider building Twitter and Reddit scrapers, but you open yourself to new problems: writing and scaling the scrapers, handling the security issues, etc. Plus, it won't be as efficient as consuming API endpoints. Looking at the current open-source Twitter scrapers (e.g. Twint or twitter-scraper), they don't seem to work anymore after the recent updates made in the Twitter API.

As an alternative to OpenAI, we can provide the user with a way to set up a connexion with an open-source model, that we serve ourselves. I don't know if the results will be as good though.

Regarding OpenAI cost, we can control it by printing the running cost each time a response is displayed. However, computing the running cost doesn't work if the response messages are streamed from OpenAI. This limits the user experience a bit.