openai-cookbook icon indicating copy to clipboard operation
openai-cookbook copied to clipboard

Delayed completion for embeddings

Open Avs-safety opened this issue 2 years ago • 5 comments

Hi,

Is it possible to have example code to add a delayed completion for producing embeddings please?

I've read 'How to handle rate limits', however due to a lack of knowledge on my part I'm struggling to add a delayed completion function to the code below.

import openai
from openai.embeddings_utils import get_embedding

size = 'babbage'

df['embeddings'] = df['Narrative Description'].apply(lambda x: get_embedding(x, engine=f'text-search-{size}-doc-001'))

df.head()

Thank you in advance!

Avs-safety avatar Jan 18 '23 16:01 Avs-safety

Yes, I've written an example script and will share it later today (or Monday at the latest).

ted-at-openai avatar Jan 20 '23 17:01 ted-at-openai

That would be great, thank you!

Avs-safety avatar Jan 24 '23 13:01 Avs-safety

Here's an example parallel processing script that will automatically throttle to stay under rate limits: https://github.com/openai/openai-cookbook/blob/main/examples/api_request_parallel_processor.py

ted-at-openai avatar Jan 24 '23 21:01 ted-at-openai

which python works for example? many error message happened when i was running the codes.

Datamore avatar Jan 29 '23 04:01 Datamore

I used Python 3.9. Apologies for the lack of versioning - I know that type hints especially can break 3.7 and earlier.

ted-at-openai avatar Feb 06 '23 17:02 ted-at-openai