alexrudall
Validated Schema
Is your feature request related to a problem? Please describe.
I'm the maintainer of Instructor which uses pydantic (python's validation) library to improve the sdk usaibility. If I wanted to do something like this in ruby, do you recommend ruby-openai support it? or should there be some seperate library that patchs this one, and adds the various new keywords?
In python it looks like
import instructor
from openai import OpenAI
from pydantic import BaseModel
# This enables response_model keyword
# from client.chat.completions.create
client = instructor.patch(OpenAI())
class UserDetail(BaseModel):
name: str
age: int
user = client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=UserDetail,
messages=[
{"role": "user", "content": "Extract Jason is 25 years old"},
]
)
assert isinstance(user, UserDetail)
assert user.name == "Jason"
assert user.age == 25
In ruby you can imagine using
UserSchema = Dry::Validation.Schema do
required(:name).filled(:str?)
required(:age).filled(:int?, gt?: 0)
end
response = client.chat(
parameters: {
model: "gpt-3.5-turbo", # Required.
messages: [{ role: "user", content: "Extract Jason is 25 years old"}], # Required.
temperature: 0.7,
},
response_model: UserSchema
)
if response.success?
# The response is valid, assert each attribute
user = validation_result.output
raise 'Name is incorrect' unless user[:name] == "Jason"
raise 'Age is incorrect' unless user[:age] == 25
Instructor is mostly a bunch of docs on how to 'think' about the idea. like: https://jxnl.github.io/instructor/concepts/prompting/
Somewhat tangentially - I personally tend to favour encapsulating the API call inside a factory method on a PORO domain class that then builds an instance from the response. I then use ActiveModel::Validations for validations on that instance. Something like:
user = User.extract_from('Jason is 25 years old')
if user.valid?
puts user.name, user.age
else
puts user.errors.full_messages.join("\n") user.valid?
end
Somewhat tangentially - I personally tend to favour encapsulating the API call inside a factory method on a PORO domain class that then builds an instance from the response. I then use
ActiveModel::Validationsfor validations on that instance. Something like:user = User.extract_from('Jason is 25 years old') if user.valid? puts user.name, user.age else puts user.errors.full_messages.join("\n") user.valid? end
ok, you should check out marvin i think they have the cleanest version of that.
I think its importnat to give the users constrol of the whole messages array.
@jxnl seems like ruby-openai would be able to support it with some patching. Although I wouldn't use dry-validation. I would use ActiveModel.
Something else to consider is that these libs don't output json schema so that'd need to be added... I think.
btw fantastic work you're doing with Instructor. I'm following closely. Here's how it might look like in Ruby:
class UserDetail
include ActiveModel::Attributes
attribute :name, :string
attribute :age, :integer
end
client = OpenAI::Client.new
user = client.chat(
parameters: {
model: "gpt-3.5-turbo",
response_model: UserDetail,
messages: [{ "role": "user", "content": "Extract Jason is 25 years old" }]
}
)
RSpec.describe "Attribute Assignment" do
it "assigns a value to an attribute" do
expect(user).to be_instance_of(UserDetail)
expect(user.name).to eq("Jason")
expect(user.age).to eq(25)
end
end
I think that would be awesome. good go everyone.
@jxnl seems like ruby-openai would be able to support it with some patching. Although I wouldn't use dry-validation. I would use ActiveModel.
Something else to consider is that these libs don't output json schema so that'd need to be added... I think.
btw fantastic work you're doing with Instructor. I'm following closely. Here's how it might look like in Ruby:
class UserDetail include ActiveModel::Attributes attribute :name, :string attribute :age, :integer end client = OpenAI::Client.new user = client.chat( parameters: { model: "gpt-3.5-turbo", response_model: UserDetail, messages: [{ "role": "user", "content": "Extract Jason is 25 years old" }] } ) RSpec.describe "Attribute Assignment" do it "assigns a value to an attribute" do expect(user).to be_instance_of(UserDetail) expect(user.name).to eq("Jason") expect(user.age).to eq(25) end end
That would be awesome. I'm not much of a Ruby developer, but I think it could benefit a lot of folks. We've had a lot of progress on the JavaScript side.
@jxnl I'd like to take a stab at a solution for Ruby for the problem you are solving with Instructor. Would you be interested in collaborating?