Sam Toyer

If you want a more precise (but not perfectly exact) way to check, you can do something like this, which parses progressively larger portions of the input. This is much...

Currently on hiatus b/c OpenAI's 1.x library is too buggy to use in prod (memory leaks, async hangs after $n$ calls, etc.).

Oh you're right, [it's not there at all](https://developers.generativeai.google/api/python/google/generativeai/chat). For some reason I thought Google had this, but I guess not.

(I checked Anthropic and DeepInfra, and they seem to be in the same boat.)

GPT 3.5 and 4 are the most important. It's okay if we're missing the rest!

(Tiffany changed the title so that is all _other_ models than OpenAI)

> Is the table too cumbersome? Is there any other design/formating that we can use? Possibly? I'd be inclined to hide it behind a button. Further styling fixes can wait...

Real stats from a staging instance: ``` { "insertId": "64d168cf000767a84a93d96c", "jsonPayload": { "memory_usage": { "used": "695MB", "free": "328MB", "total": "1024MB" }, "uptime_and_load": { "uptime": "10 hours", "load_avg": [ "0.00,", "0.00",...

How caching works in Django: https://docs.djangoproject.com/en/4.2/topics/cache/ Note that you can cache specific fragments of templates that are expensive to render, or cache entire views. We would configure Django to use...

Caching of OpenAI requests might be a bit more complicated: it might have to be done at a lower level, and possibly needs to be saved to disk so that...