cohere-python icon indicating copy to clipboard operation
cohere-python copied to clipboard

finetuned model ID mismatch between Python SDK and UI

Open mdelrosa opened this issue 1 year ago • 1 comments

Issue: When using the Python SDK to finetune a classification model, the returned ID does not match the correct model ID. More specifically, the ID in the Python SDK resembles ``, but the ID shown in the UI (i.e., the correct ID) resembles -ft.

Solution: have the GetFinetunedModelResponse in the Python SDK return the "correct" model ID that matches the ID shown in the UI.

Steps to Reproduce: To reproduce/get the ID in the Python SDK, you can do the following:

import cohere

COHERE_API_KEY = "YOUR_API_KEY"
co = cohere.Client(api_key=COHERE_API_KEY)

model = co.finetuning.create_finetuned_model(
	request=FinetunedModel(
		name="my-model-name",
		settings=Settings(
			base_model=BaseModel(
				base_type="BASE_TYPE_CLASSIFICATION",
			),
			dataset_id="my-dataset-id",
		),
	)
)
model_id = model.finetuned_model.id

clf_response = co.classify(
    inputs=["classify this!"],
    model=model_id
)

The co.classify call fails with the following error:

ApiError: status_code: 404, body: {'message': "model '<MODEL_UUID>' not found, make sure the correct model ID was used and that you have access to the model."}"

If we navigate to the UI, we find that the `` for the finetuned model has a "-ft" suffix, so the user can get around the ApiError just by running:

clf_response = co.classify(
    inputs=["classify this!"],
    model=model_id+"-ft"
)

mdelrosa avatar May 03 '24 19:05 mdelrosa

Same problem here

JulianArruti avatar May 04 '24 03:05 JulianArruti