openai-cookbook
openai-cookbook copied to clipboard
Add LLMOps folder and W&B + OpenAI integration quickstart
Hello! I work at Weights and Biases and we have many offerings/teachings related to OpenAI as we work closely with them. Our MLOps platform has many first class integrations to for usage with OpenAI that we would love to show the community.
The W&B OpenAI AutoLogger logs requests, responses, token counts and model metadata with 1 line of code for all OpenAI models, allowing users to operationalize their LLM workflows.
Let me know if there are any suggestions or changes!
Hey @ted-at-openai! I was wondering if there were any other changes I needed to make
Thanks for the ping. I will take a look. (Apologies for the slow pace on this repo.)
Ok, going through the notebook now. Immediately hit an error in the first cell:
Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving.
wandb: Currently logged in as: {username}. Use `wandb login --relogin` to force relogin
wandb: ERROR Error while calling W&B API: permission denied (<Response [403]>)
Problem at: [path] 108 _run_init
---------------------------------------------------------------------------
CommError Traceback (most recent call last)
...
CommError: It appears that you do not have permission to access the requested resource. Please reach out to the project owner to grant you access. If you have the correct permissions, verify that there are no issues with your networking setup.(Error 403: Forbidden)
Not obvious to me what I did wrong or how to fix it, so I expect other users might also get confused here. Are users supposed to enter a pre-existing llm project name?
In the meantime, I've added a link on the Cookbook homepage advertising W&B as "A paid product for tracking model training and prompt engineering experiments." Hope this helps people discover what you're building.
One thought: This notebook looks pretty brief. Do you think it would make sense to discuss the value of logging inputs and outputs? Or a scenario in which a user might benefit from the logs? Right now I worry people might stumble upon it, run the ABC process of (a) turn logger on, (b) call API, (c) turn logger off, without really understanding what that might be useful to them.
Still want to add this? In the meantime, we've added a folder third_party_examples
where this would fit nicely.
#714 Hey @ted-at-openai, I decided to open a new PR to add W&B examples to the third_party_examples
you mentioned.