openai-python
openai-python copied to clipboard
Migrating to v1: How to create a fine-tuning job and stream using async client?
Confirm this is an issue with the Python library and not an underlying OpenAI API
- [X] This is an issue with the Python library
Describe the bug
In v0, astream_events
is able to iterate and wait until the model is actually completed.
I've migrated it to v1 (see snippets), but fine_tuning.jobs.list_events()
doesn't wait until the model is actually completed. The messages also seem to be in the reverse order.
I know that this is because it's not streaming, but I don't see a method or parameter (e.g. stream=True
) that allows me to stream and wait for the events.
To Reproduce
- Use AsyncClient
- await fine tuning job create
- try to stream and wait for all events
Code snippets
# v0 - works
with open('ai_data/model.jsonl', 'rb') as f:
upload_resp = await openai.File.acreate(f, 'fine-tune')
file_id = upload_resp.id
tune_resp = await openai.FineTune.acreate(training_file=file_id, model='babbage')
async for event in await openai.FineTune.astream_events(tune_resp.id):
print(event.message)
print('Completed. New Model ID: ' + tune_resp.fine_tuned_model)
# v1
with open('ai_data/model.jsonl', 'rb') as f:
upload_resp = await client.files.create(file=f, purpose='fine-tune')
file_id = upload_resp.id
tune_resp = await client.fine_tuning.jobs.create(training_file=file_id, model='babbage-002')
async for event in client.fine_tuning.jobs.list_events(tune_resp.id):
print(event.message)
# Only prints the following
# Sending model train request
# Validating training file: file-XZpR8yio1o1Hw7wPyxfRXtYQ
# Created fine-tuning job: ftjob-f9UuWfBcOOu3Vhd0jR3k65oa
print('Completed. New Model ID: ' + tune_resp.fine_tuned_model) # Error because the model is not completed yet
OS
Windows
Python version
3.11.5
Library version
1.10.6