fix(deps): update dependency openai to v4.100.0
This PR contains the following updates:
| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| openai | 4.96.2 -> 4.100.0 |
Release Notes
openai/openai-node (openai)
v4.100.0
Full Changelog: v4.99.0...v4.100.0
Features
- api: further updates for evals API (3f6f248)
Chores
- internal: version bump (5123fe0)
v4.99.0
Full Changelog: v4.98.0...v4.99.0
Features
- api: manual updates (75eb804)
- api: responses x eval api (5029f1a)
- api: Updating Assistants and Evals API schemas (27fd517)
v4.98.0
Full Changelog: v4.97.0...v4.98.0
Features
- api: Add reinforcement fine-tuning api support (4aa7a79)
Chores
Documentation
- add examples to tsdocs (7d841b7)
v4.97.0
Full Changelog: v4.96.2...v4.97.0
Features
- api: add image sizes, reasoning encryption (9c2113a)
Chores
- docs: add missing deprecation warnings (253392c)
Documentation
Configuration
📅 Schedule: Branch creation - Tuesday through Thursday ( * * * * 2-4 ) (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
- [ ] If you want to rebase/retry this PR, check this box
This PR was generated by Mend Renovate. View the repository job log.
anthropic debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..4e19f03d6 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.97.0"
}
diff --git .stats.yml .stats.yml
index d92408173..0c8278866 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-0ee6b36cf3cc278cef4199a6aec5f7d530a6c1f17a74830037e96d50ca1edc50.yml
+openapi_spec_hash: e8ec5f46bc0655b34f292422d58a60f6
+config_hash: d9b6b6e6bc85744663e300eebc482067
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..6fa637742 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,24 @@
# Changelog
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..4175b4a71 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -57,4 +57,4 @@ async function main() {
rt.socket.addEventListener('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..e86a79092 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -57,4 +57,4 @@ async function main() {
rt.socket.on('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..f1c46dd41 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -45,4 +45,4 @@ async function main() {
rt.socket.addEventListener('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..1ce6b2045 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -45,4 +45,4 @@ async function main() {
rt.socket.on('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git jsr.json jsr.json
index 8eca06e74..fd3ca4a41 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.97.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..8b9281b35 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.97.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..e218c8299 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -66,7 +66,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..251020337 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -1169,7 +1169,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1187,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1220,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..2198e8174 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -466,8 +466,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..32f1e123c 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -162,10 +162,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +178,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +229,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..0a6e3666d 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -158,7 +158,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +209,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +258,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1581,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1656,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1673,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1764,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1792,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1824,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1845,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2125,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2145,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2744,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2800,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2811,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2830,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2885,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/version.ts src/version.ts
index 1674d74fe..97cbc5900 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.97.0'; // x-release-please-version
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This PR updates the OpenAI Node SDK from version 4.96.2 to 4.97.0, incorporating new features, API changes, and bug fixes. The changes include support for additional image sizes, reasoning encryption functionality, and various documentation improvements including fixing typos and clarifying deprecation warnings.
Changes
Changes
-
Version Bumps and Changelog
- Updated version from 4.96.2 to 4.97.0 across multiple files (.release-please-manifest.json, jsr.json, package.json, src/version.ts)
- Added new changelog entries documenting features, chores, and documentation improvements
-
Documentation Improvements
- Fixed typo in README.md ("as followed" → "as follows")
- Fixed multiple instances of "procesing" → "processing" in examples and documentation
- Added missing @deprecated annotations to better highlight deprecated parameters
-
API Updates
-
Images API:
- Updated ImageEditParams to support transparency settings with a new
backgroundparameter - Added support for new image sizes (1536x1024, 1024x1536)
- Improved documentation for image editing capabilities
- Updated ImageEditParams to support transparency settings with a new
-
Speech API:
- Added note that speech speed parameter doesn't work with
gpt-4o-mini-tts
- Added note that speech speed parameter doesn't work with
-
Responses API:
- Added
reasoning.encrypted_contentto ResponseIncludable to support multi-turn conversations - Updated ComputerTool environment types to include 'linux'
- Made several parameter types more flexible by adding null options
- Enhanced the ResponseReasoningItem interface with encrypted_content
- Added
-
-
Tests
- Updated image edit test to include the new
backgroundparameter
- Updated image edit test to include the new
sequenceDiagram
participant Dev as Developer
participant SDK as OpenAI SDK
participant API as OpenAI API
Dev->>SDK: Update to v4.97.0
Note over SDK: Added support for image transparency
Note over SDK: Added reasoning encryption
Note over SDK: Fixed documentation typos
Dev->>SDK: images.edit() with background: 'transparent'
SDK->>API: POST /images/edits with background param
API-->>SDK: Return edited image with transparency
SDK-->>Dev: Return response with edited image
Dev->>SDK: Create response with reasoning.encrypted_content
SDK->>API: POST /responses with include=[reasoning.encrypted_content]
API-->>SDK: Return response with encrypted reasoning
SDK-->>Dev: Return response object with encrypted reasoning field
openai debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..4e19f03d6 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.97.0"
}
diff --git .stats.yml .stats.yml
index d92408173..0c8278866 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-0ee6b36cf3cc278cef4199a6aec5f7d530a6c1f17a74830037e96d50ca1edc50.yml
+openapi_spec_hash: e8ec5f46bc0655b34f292422d58a60f6
+config_hash: d9b6b6e6bc85744663e300eebc482067
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..6fa637742 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,24 @@
# Changelog
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..4175b4a71 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -57,4 +57,4 @@ async function main() {
rt.socket.addEventListener('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..e86a79092 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -57,4 +57,4 @@ async function main() {
rt.socket.on('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..f1c46dd41 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -45,4 +45,4 @@ async function main() {
rt.socket.addEventListener('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..1ce6b2045 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -45,4 +45,4 @@ async function main() {
rt.socket.on('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git jsr.json jsr.json
index 8eca06e74..fd3ca4a41 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.97.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..8b9281b35 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.97.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..e218c8299 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -66,7 +66,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..251020337 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -1169,7 +1169,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1187,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1220,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..2198e8174 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -466,8 +466,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..32f1e123c 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -162,10 +162,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +178,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +229,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..0a6e3666d 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -158,7 +158,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +209,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +258,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1581,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1656,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1673,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1764,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1792,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1824,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1845,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2125,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2145,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2744,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2800,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2811,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2830,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2885,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/version.ts src/version.ts
index 1674d74fe..97cbc5900 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.97.0'; // x-release-please-version
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This pull request updates the OpenAI Node.js library from version 4.96.2 to 4.97.0. It includes several fixes, enhancements, and documentation updates. Key changes include new features like image size options and reasoning encryption, as well as various deprecations and improvements to error messages and documentation.
Security Hotspots
None identified.
Changes
Changes
General Configuration Updates
-
.release-please-manifest.json
- ".": "4.96.2" + ".": "4.97.0" -
.stats.yml
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml +openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-0ee6b36cf3cc278cef4199a6aec5f7d530a6c1f17a74830037e96d50ca1edc50.yml -openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03 +openapi_spec_hash: e8ec5f46bc0655b34f292422d58a60f6 -config_hash: b597cd9a31e9e5ec709e2eefb4c54122 +config_hash: d9b6b6e6bc85744663e300eebc482067
Changelog Updates
-
CHANGELOG.md
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb)) +* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16)) +* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f)) +* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
Realtime Examples Typos Fix
-
README.md
-Error codes are as followed: +Error codes are as follows: -
realtime.md (Multiple files have this typo correction)
-// likely want to continue procesing events regardless of any errors +// likely want to continue processing events regardless of any errors
Source Code Enhancements
-
src/resources/audio/speech.ts
+*speed option now includes a note that it does not work with `gpt-4o-mini-tts`. -
src/resources/chat/completions/completions.ts
+ * added @deprecated annotations for `function_call`, `functions` and `max_tokens`. -
src/resources/fine-tuning/jobs/jobs.ts
+ * added @deprecated annotations for `hyperparameters`. -
src/resources/images.ts
+ * added support for new image sizes and background transparency options for `gpt-image-1`. -
src/resources/responses/responses.ts
+ * multiple enhancements, including nullable types, new `reasoning.encrypted_content` option, and handling additional fields for `ComputerTool`, `FileSearchTool`, and `FunctionTool`. -
src/version.ts
-export const VERSION = '4.96.2'; +export const VERSION = '4.97.0';
Test Enhancements
-
tests/api-resources/images.test.ts
+ * added a test case to include the new background transparency parameter.
sequenceDiagram
participant Dev as Developer
participant Repo as Repository
participant CI as Continuous Integration
Dev->>Repo: Creates a new branch with changes
Dev->>Repo: Opens Pull Request
Repo-->>CI: Notifies CI for testing
CI-->>Repo: Reports test and lint results
Repo-->>Dev: Allows code review and approval
bedrock debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..4e19f03d6 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.97.0"
}
diff --git .stats.yml .stats.yml
index d92408173..0c8278866 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-0ee6b36cf3cc278cef4199a6aec5f7d530a6c1f17a74830037e96d50ca1edc50.yml
+openapi_spec_hash: e8ec5f46bc0655b34f292422d58a60f6
+config_hash: d9b6b6e6bc85744663e300eebc482067
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..6fa637742 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,24 @@
# Changelog
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..4175b4a71 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -57,4 +57,4 @@ async function main() {
rt.socket.addEventListener('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..e86a79092 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -57,4 +57,4 @@ async function main() {
rt.socket.on('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..f1c46dd41 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -45,4 +45,4 @@ async function main() {
rt.socket.addEventListener('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..1ce6b2045 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -45,4 +45,4 @@ async function main() {
rt.socket.on('close', () => console.log('\nConnection closed!'));
}
-main();
+main();
\ No newline at end of file
diff --git jsr.json jsr.json
index 8eca06e74..fd3ca4a41 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.97.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..8b9281b35 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.97.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..e218c8299 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -66,7 +66,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..251020337 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -1169,7 +1169,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1187,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1220,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..2198e8174 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -466,8 +466,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..32f1e123c 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -162,10 +162,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +178,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +229,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..0a6e3666d 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -158,7 +158,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +209,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +258,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1581,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1656,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1673,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1764,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1792,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1824,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1845,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2125,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2145,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2744,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2800,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2811,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2830,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2885,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/version.ts src/version.ts
index 1674d74fe..97cbc5900 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.97.0'; // x-release-please-version
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This PR updates the OpenAI Node.js SDK from version 4.96.2 to 4.97.0. It introduces new features like additional image sizes and reasoning encryption, fixes typos in documentation, and enhances API parameters with better descriptions and deprecation warnings.
Changes
Changes
-
Version Updates:
- Updated version from 4.96.2 to 4.97.0 in multiple files (.release-please-manifest.json, package.json, jsr.json, src/version.ts)
- Added CHANGELOG entries for version 4.97.0
-
Documentation Improvements:
- Fixed a typo in README.md: "Error codes are as followed" → "Error codes are as follows"
- Fixed typos in realtime examples: "procesing" → "processing" in multiple files
- Added missing deprecation warnings using proper JSDoc syntax
-
API Enhancements:
-
Speech API: Added clarification that speed parameter doesn't work with
gpt-4o-mini-tts -
Images API:
- Added support for new image sizes (1536x1024, 1024x1536)
- Added new
backgroundparameter for image edits withtransparent,opaque, orautooptions - Updated image edit documentation to clarify format requirements
-
Responses API:
- Added
reasoning.encrypted_contentto possible includes for responses - Enhanced ComputerTool environment types to include 'linux'
- Made several parameter types nullable where appropriate
- Added
-
Chat Completions API:
- Added proper JSDoc deprecation tags to deprecated parameters
-
Speech API: Added clarification that speed parameter doesn't work with
-
Test Updates:
- Added
background: 'transparent'parameter to the image edit test
- Added
sequenceDiagram
participant User
participant SDK
participant OpenAI
Note over User,OpenAI: Version 4.97.0 changes
User->>SDK: Use new image sizes (1536x1024/1024x1536)
SDK->>OpenAI: Send request with new sizes
OpenAI-->>SDK: Return images with requested dimensions
User->>SDK: Set background: 'transparent' for image edits
SDK->>OpenAI: Request with transparent background
OpenAI-->>SDK: Return image with transparent background
User->>SDK: Include 'reasoning.encrypted_content'
SDK->>OpenAI: Request response with encrypted reasoning
OpenAI-->>SDK: Return response with encrypted reasoning content
User->>SDK: Use response for stateless conversation
SDK->>OpenAI: Send encrypted reasoning in follow-up message
OpenAI-->>SDK: Continue conversation with context
Note over User,OpenAI: Documentation improvements
User->>SDK: Read fixed documentation
SDK-->>User: Provide more accurate information
anthropic debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..a279d9124 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.98.0"
}
diff --git .stats.yml .stats.yml
index d92408173..5f1bee851 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-794a6ed3c3d3d77887564755168056af8a426b17cf1ec721e3a300503dc22a41.yml
+openapi_spec_hash: 25a81c220713cd5b0bafc221d1dfa79a
+config_hash: 0b768ed1b56c6d82816f0fa40dc4aaf5
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..2f1a39177 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,43 @@
# Changelog
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git api.md api.md
index 49e6548a8..cad696e7e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -669,10 +709,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..25bbc9ac2 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.98.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..d34efceb0 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.98.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..8d563e0ba 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..61e58a8e8 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..cc040abff 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..0f7166df4 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -103,83 +104,6 @@ export interface EvalCustomDataSourceConfig {
type: 'custom';
}
-/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
/**
* A StoredCompletionsDataSourceConfig which specifies the metadata property of
* your stored completions query. This is usually metadata like `usecase=chatbot`
@@ -210,83 +134,6 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
@@ -335,39 +182,29 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
- /**
- * The name of the grader.
- */
- name: string;
-
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +214,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -507,39 +270,29 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +302,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -679,39 +358,29 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +390,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -851,39 +446,29 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +478,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -987,15 +498,15 @@ export interface EvalCreateParams {
/**
* The configuration for the data source used for the evaluation runs.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.StoredCompletions;
/**
* A list of graders for all eval runs in this group.
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1048,14 +559,14 @@ export namespace EvalCreateParams {
* completions query. This is usually metadata like `usecase=chatbot` or
* `prompt-version=v2`, etc.
*/
- export interface Logs {
+ export interface StoredCompletions {
/**
- * The type of data source. Always `logs`.
+ * The type of data source. Always `stored_completions`.
*/
- type: 'logs';
+ type: 'stored_completions';
/**
- * Metadata filters for the logs data source.
+ * Metadata filters for the stored completions data source.
*/
metadata?: Record<string, unknown>;
}
@@ -1154,29 +665,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +687,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +733,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha }, from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..08616cd4f 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
- */
- dpo?: Method.Dpo;
-
- /**
- * Configuration for the supervised fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- supervised?: Method.Supervised;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..9ee08f75f
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..1440e865e 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -61,6 +61,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +98,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +125,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +180,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +231,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +280,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1603,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1678,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1695,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1786,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1814,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1846,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1867,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2147,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2167,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2766,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2822,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2833,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2852,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2907,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/version.ts src/version.ts
index 1674d74fe..f64cc03ff 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.98.0'; // x-release-please-version
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This PR updates the OpenAI Node.js SDK with new features, enhancements, and bug fixes. The primary updates include:
- Adding support for reinforcement fine-tuning API
- Updating the Node.js version in CI workflows from 18 to 20
- Adding comprehensive JSDoc examples to improve documentation
- Fixing typos and improving error messages
- Restructuring the graders API by moving grader types into their own namespace
- Updating version numbers from 4.96.2 to 4.98.0
Changes
Changes
GitHub Workflows
- Updated Node.js version from '18' to '20' across all CI and publishing workflows
Documentation
- Added JSDoc examples for most API methods to improve developer experience
- Fixed typo in README.md: "Error codes are as followed" → "Error codes are as follows"
- Fixed typo in realtime examples: "procesing" → "processing"
- Added comprehensive documentation for the new reinforcement fine-tuning features
API Updates
- Added reinforcement fine-tuning API support
- Added pause and resume methods to fine-tuning jobs
- Added a new graders resource with various grader types
- Updated the Images API to support new image sizes and transparency controls
- Added support for reasoning encryption in chat completions
- Marked deprecated parameters with the @deprecated JSDoc tag
- Restructured the way grader models are defined in the codebase
Fine-tuning and Method Files
- Created new method types for different fine-tuning approaches
- Added DPO, Reinforcement, and Supervised method types
- Added support for alpha graders in the fine-tuning API
Version Updates
- Updated version from 4.96.2 to 4.98.0 in:
- .release-please-manifest.json
- jsr.json
- package.json
- CHANGELOG.md (with detailed release notes)
sequenceDiagram
Developer->>OpenAI SDK: Create fine-tuning job with new reinforcement method
OpenAI SDK->>API: POST /fine_tuning/jobs with reinforcement parameters
API->>OpenAI SDK: Return FineTuningJob object
Developer->>OpenAI SDK: Pause fine-tuning job
OpenAI SDK->>API: POST /fine_tuning/jobs/{id}/pause
API->>OpenAI SDK: Return updated FineTuningJob object
Developer->>OpenAI SDK: Resume fine-tuning job
OpenAI SDK->>API: POST /fine_tuning/jobs/{id}/resume
API->>OpenAI SDK: Return updated FineTuningJob object
Developer->>OpenAI SDK: Use grader to evaluate model output
OpenAI SDK->>API: POST /fine_tuning/alpha/graders/run
API->>OpenAI SDK: Return grader evaluation results
bedrock debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..a279d9124 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.98.0"
}
diff --git .stats.yml .stats.yml
index d92408173..5f1bee851 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-794a6ed3c3d3d77887564755168056af8a426b17cf1ec721e3a300503dc22a41.yml
+openapi_spec_hash: 25a81c220713cd5b0bafc221d1dfa79a
+config_hash: 0b768ed1b56c6d82816f0fa40dc4aaf5
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..2f1a39177 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,43 @@
# Changelog
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git api.md api.md
index 49e6548a8..cad696e7e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -669,10 +709,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..25bbc9ac2 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.98.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..d34efceb0 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.98.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..8d563e0ba 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..61e58a8e8 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..cc040abff 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..0f7166df4 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -103,83 +104,6 @@ export interface EvalCustomDataSourceConfig {
type: 'custom';
}
-/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
/**
* A StoredCompletionsDataSourceConfig which specifies the metadata property of
* your stored completions query. This is usually metadata like `usecase=chatbot`
@@ -210,83 +134,6 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
@@ -335,39 +182,29 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
- /**
- * The name of the grader.
- */
- name: string;
-
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +214,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -507,39 +270,29 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +302,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -679,39 +358,29 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +390,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -851,39 +446,29 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +478,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -987,15 +498,15 @@ export interface EvalCreateParams {
/**
* The configuration for the data source used for the evaluation runs.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.StoredCompletions;
/**
* A list of graders for all eval runs in this group.
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1048,14 +559,14 @@ export namespace EvalCreateParams {
* completions query. This is usually metadata like `usecase=chatbot` or
* `prompt-version=v2`, etc.
*/
- export interface Logs {
+ export interface StoredCompletions {
/**
- * The type of data source. Always `logs`.
+ * The type of data source. Always `stored_completions`.
*/
- type: 'logs';
+ type: 'stored_completions';
/**
- * Metadata filters for the logs data source.
+ * Metadata filters for the stored completions data source.
*/
metadata?: Record<string, unknown>;
}
@@ -1154,29 +665,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +687,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +733,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha }, from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..08616cd4f 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
- */
- dpo?: Method.Dpo;
-
- /**
- * Configuration for the supervised fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- supervised?: Method.Supervised;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..9ee08f75f
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..1440e865e 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -61,6 +61,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +98,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +125,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +180,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +231,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +280,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1603,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1678,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1695,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1786,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1814,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1846,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1867,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2147,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2167,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2766,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2822,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2833,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2852,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2907,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/version.ts src/version.ts
index 1674d74fe..f64cc03ff 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.98.0'; // x-release-please-version
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This PR updates the OpenAI Node.js client library from version 4.96.2 to 4.98.0, introducing support for reinforcement fine-tuning, adding grader models, implementing pause/resume functionality for fine-tuning jobs, and upgrading Node.js from version 18 to 20 in the CI workflows. The PR also includes numerous documentation improvements by adding JSDoc examples to API methods, fixing typos, and clarifying deprecation warnings.
Changes
Changes
CI Workflows
- Updated Node.js version from 18 to 20 in all GitHub workflow files (ci.yml, publish-jsr.yml, publish-npm.yml)
Documentation
- Added JSDoc examples to various API methods
- Fixed typo in README.md: "Error codes are as followed" → "Error codes are as follows"
- Fixed multiple instances of "procesing" → "processing" in realtime examples
- Improved API documentation in api.md to include new methods and types
- Formatted examples/tsconfig.json
Version Updates
- Bumped version from 4.96.2 to 4.98.0 in package.json, jsr.json, .release-please-manifest.json
- Updated CHANGELOG.md with entries for versions 4.97.0 and 4.98.0
API Enhancements
-
Fine-Tuning:
- Added reinforcement fine-tuning API support
- Implemented new pause and resume methods for fine-tuning jobs
- Created new method types (supervised, DPO, reinforcement) with appropriate hyperparameters
- Added alpha graders functionality for fine-tuning
-
Graders:
- Added new Graders resource with various grader models:
- LabelModelGrader
- MultiGrader
- PythonGrader
- ScoreModelGrader
- StringCheckGrader
- TextSimilarityGrader
- Moved grader types from Evals to shared Graders resources
- Added new Graders resource with various grader models:
-
Images API:
- Added support for new image sizes
- Added background transparency option for image edits
-
Responses API:
- Added support for reasoning encryption
- Added more environments to computer tool
sequenceDiagram
participant Client as Client Application
participant SDK as OpenAI Node.js SDK
participant API as OpenAI API
note over Client, API: Node.js upgrade from v18 to v20
Client->>SDK: Use new reinforcement fine-tuning API
SDK->>API: Create fine-tuning job with reinforcement settings
Client->>SDK: Pause fine-tuning job
SDK->>API: POST /fine_tuning/jobs/{id}/pause
API-->>SDK: Return paused job status
Client->>SDK: Resume fine-tuning job
SDK->>API: POST /fine_tuning/jobs/{id}/resume
API-->>SDK: Return resumed job status
Client->>SDK: Use grader models
SDK->>API: Interact with grader endpoints
Client->>SDK: Generate images with new sizes
SDK->>API: Request image generation with new parameters
API-->>SDK: Return generated images
Client->>SDK: Create image edits with transparency
SDK->>API: Request image edits with transparency settings
API-->>SDK: Return edited images
Client->>SDK: Generate responses with reasoning encryption
SDK->>API: Request with reasoning.encrypted_content
API-->>SDK: Return encrypted reasoning content
openai debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..a279d9124 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.98.0"
}
diff --git .stats.yml .stats.yml
index d92408173..5f1bee851 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-794a6ed3c3d3d77887564755168056af8a426b17cf1ec721e3a300503dc22a41.yml
+openapi_spec_hash: 25a81c220713cd5b0bafc221d1dfa79a
+config_hash: 0b768ed1b56c6d82816f0fa40dc4aaf5
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..2f1a39177 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,43 @@
# Changelog
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git api.md api.md
index 49e6548a8..cad696e7e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -669,10 +709,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..25bbc9ac2 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.98.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..d34efceb0 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.98.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..8d563e0ba 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..61e58a8e8 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..cc040abff 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..0f7166df4 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -103,83 +104,6 @@ export interface EvalCustomDataSourceConfig {
type: 'custom';
}
-/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
/**
* A StoredCompletionsDataSourceConfig which specifies the metadata property of
* your stored completions query. This is usually metadata like `usecase=chatbot`
@@ -210,83 +134,6 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
@@ -335,39 +182,29 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
- /**
- * The name of the grader.
- */
- name: string;
-
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +214,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -507,39 +270,29 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +302,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -679,39 +358,29 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +390,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -851,39 +446,29 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +478,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -987,15 +498,15 @@ export interface EvalCreateParams {
/**
* The configuration for the data source used for the evaluation runs.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.StoredCompletions;
/**
* A list of graders for all eval runs in this group.
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1048,14 +559,14 @@ export namespace EvalCreateParams {
* completions query. This is usually metadata like `usecase=chatbot` or
* `prompt-version=v2`, etc.
*/
- export interface Logs {
+ export interface StoredCompletions {
/**
- * The type of data source. Always `logs`.
+ * The type of data source. Always `stored_completions`.
*/
- type: 'logs';
+ type: 'stored_completions';
/**
- * Metadata filters for the logs data source.
+ * Metadata filters for the stored completions data source.
*/
metadata?: Record<string, unknown>;
}
@@ -1154,29 +665,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +687,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +733,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha }, from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..08616cd4f 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
- */
- dpo?: Method.Dpo;
-
- /**
- * Configuration for the supervised fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- supervised?: Method.Supervised;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..9ee08f75f
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..1440e865e 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -61,6 +61,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +98,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +125,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +180,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +231,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +280,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1603,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1678,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1695,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1786,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1814,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1846,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1867,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2147,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2167,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2766,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2822,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2833,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2852,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2907,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/version.ts src/version.ts
index 1674d74fe..f64cc03ff 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.98.0'; // x-release-please-version
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This pull request updates multiple areas of the codebase with new features, enhancements, and maintenance updates. The primary changes include:
- Node Version Update: All CI/CD workflows have been updated to use Node.js v20 instead of v18.
- New Feature: Added support for reinforcement fine-tuning in the API.
-
API Enhancements:
- Methods for pausing and resuming fine-tuning jobs.
- Additional endpoints for fine-tuning operations.
- Grader APIs: Introduced several new grader models and associated API methods for validation and execution.
- Documentation Updates: Lots of new examples added across the documentation in JSDoc format.
- Bug Fixes and Improvements: Minor bug fixes, typo corrections, and formatting changes.
-
Version Update: Incremented the package version from
4.96.2to4.98.0.
Possible Issues
- Backward Compatibility: Users relying on Node.js v18 might face issues due to the mandatory upgrade to Node.js v20.
- Deprecations: Marked some parameters and hyperparameters as deprecated which might affect existing functionality.
Security Hotspots
There don't appear to be any direct security vulnerabilities introduced in this PR. However, ensuring the new grader APIs (used for running and validating graders) handle user inputs safely and securely is essential.
Changes
Changes
CI/CD Configuration
- Updated Node.js version from 18 to 20 in all workflow YAML files.
API Documentation
- Updated the
CHANGELOG.mdwith new feature descriptions and version updates. - Added various examples and updated JSDoc comments across multiple API classes for better clarity.
API Classes and Methods
-
API Resources:
- Added new methods for reinforcement fine-tuning.
- Included operations to pause and resume fine-tuning jobs.
- Introduced grader-related API methods under
fine-tuning/alpha.
-
Fine-Tuning:
- Introduced
methods.ts,alpha.ts, and enhancedjobs.tswith new fine-tuning configurations.
- Introduced
-
Grader Models:
- Created
grader-models.tswith definitions for new grader types. - Added validation and execution endpoints for graders in the fine-tuning alpha.
- Created
Example Files:
- Corrected typos in various example files specific to real-time processing.
- Enhanced
tsconfig.jsonfor examples directory.
Test Files
- Added tests for new grader endpoints and fine-tuning job operations.
- Updated existing tests to accommodate changes in API methods, specifically the inclusion of the new configuration types.
Please review these changes carefully to ensure they meet all requirements and functionality as intended.```mermaid sequenceDiagram participant User participant CI as CI/CD System participant API as API Gateway User->>CI: Push code changes CI->>CI: Run workflows CI->>NodeSetup: Setup Node.js v20 User->>NodeSetup: Introduce new fine-tuning method API-->>NodeSetup: Add reinforcement fine-tuning config User->>API: Request to pause fine-tune job API->>JobHandler: Pause the specified job JobHandler-->>API: Job pause confirmation User->>API: Request to resume fine-tune job API->>JobHandler: Resume the paused job JobHandler-->>API: Job resume confirmation User->>GraderAPI: Execute new grader GraderAPI-->>User: Provide grader results
</details>
<!-- Generated by gpt-4o-2024-05-13 -->
openai debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..a65cc01f9 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.99.0"
}
diff --git .stats.yml .stats.yml
index d92408173..a3c5d081d 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-5fa16b9a02985ae06e41be14946a9c325dc672fb014b3c19abca65880c6990e6.yml
+openapi_spec_hash: da3e669f65130043b1170048c0727890
+config_hash: d8d5fda350f6db77c784f35429741a2e
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..2a552b8ed 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,53 @@
# Changelog
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git api.md api.md
index 49e6548a8..cad696e7e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -669,10 +709,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..2f29927c6 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.99.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..0d756ef85 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.99.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..61e58a8e8 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..08c898ace 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -104,88 +105,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -195,9 +115,9 @@ export interface EvalStoredCompletionsDataSourceConfig {
schema: Record<string, unknown>;
/**
- * The type of data source. Always `stored_completions`.
+ * The type of data source. Always `stored-completions`.
*/
- type: 'stored_completions';
+ type: 'stored-completions';
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -210,83 +130,6 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
@@ -309,7 +152,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +181,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +243,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -481,7 +273,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +302,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +364,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -653,7 +394,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +423,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +485,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -825,7 +515,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +544,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +606,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -987,15 +626,15 @@ export interface EvalCreateParams {
/**
* The configuration for the data source used for the evaluation runs.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
* A list of graders for all eval runs in this group.
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +683,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +698,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ *, Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored-completions`.
+ */
+ type: 'stored-completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1154,29 +807,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +829,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +875,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..31883e6b5 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -356,7 +356,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +423,28 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +486,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -511,8 +505,8 @@ export namespace RunCreateResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +533,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -749,7 +748,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +815,28 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +878,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -904,8 +897,8 @@ export namespace RunRetrieveResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +925,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1139,10 +1137,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1204,28 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1267,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1297,8 +1286,8 @@ export namespace RunListResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1314,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1543,7 +1537,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1604,28 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1667,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1698,8 +1686,8 @@ export namespace RunCancelResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1714,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1955,9 +1948,9 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
input_messages?:
| CreateEvalResponsesRunDataSource.Template
@@ -2013,12 +2006,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2038,8 +2025,8 @@ export namespace RunCreateParams {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2053,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..1440e865e 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -61,6 +61,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +98,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +125,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +180,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +231,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +280,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1603,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1678,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1695,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1786,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1814,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1846,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1867,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2147,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2167,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2766,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-s,earch).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2822,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2833,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2852,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2907,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..c7ee5a162 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.99.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This pull request primarily focuses on updating the Node.js version used in CI/CD workflows from Node.js 18 to Node.js 20. Additionally, it introduces new features, enhancements, and bug fixes to the OpenAI Node.js client library.
Possible Issues
- Backward Compatibility: The Node.js version bump might lead to compatibility issues for users relying on Node.js 18 if Node.js 20 introduces breaking changes.
- Deprecations: Several deprecated features and parameters in the code might cause issues if users are not made aware of the recommended migrations.
Security Hotspots
None identified.
Changes
Changes
Workflow Configuration
- Updated Node.js version from 18 to 20 in
.github/workflows/ci.yml,.github/workflows/publish-jsr.yml, and.github/workflows/publish-npm.yml.
Module Updates
-
Changelog: Updated to reflect changes and new version
4.99.0. - README.md: Fixed typo "procesing" to "processing".
- Config files (.release-please-manifest.json, .stats.yml, jsr.json, package.json): Updated versions and endpoint configurations.
New Functionalities and Changes
- Grading and Evaluation Enhancements: Introduced new grader types and configurations with added validation and response handling.
- Fine-Tuning Methods: Added support for DPO, Supervised, and Reinforcement learning methods.
- Job Controls: Added pause and resume functionality for fine-tuning jobs.
- Image Editing: Extended functionalities for image editing, including background transparency options.
- Audio/Transcriptions: Added support for new transcription chunking strategies.
Updates to API Documentation
- API Documentation: Enhanced with more examples and use cases to guide users.
- Deprecations: Marked several parameters and configurations as deprecated and provided alternatives.
Test Cases
- Added new tests for new graders, fine-tuning job controls, and image editing functionalities.
sequenceDiagram
Note over User,Library: User interacts with OpenAI Node.js client
User->>Library: Initialize Client (v4.99.0)
Note over Library: Node 20 environment setup
User->>Library: FineTune Jobs (Create, Pause, Resume)
Note over Library: Integrates new fine-tuning methods
User->>Library: Image Editing API
Note over Library: Responds with edited image
User->>Library: Grading/Validating Inputs
Note over Library: Processes new grader configurations
User->>Library: Audio Transcriptions (Chunking)
Note over Library: Handles new chunking strategy
Library->>User: Returns API responses
Conclusion
This PR introduces notable enhancements and updates to maintain and improve the OpenAI Node.js client library. It includes Node.js version upgrades in workflows, expanded fine-tuning and grading methods, better documentation, and comprehensive tests to ensure reliability. However, developers should be aware of the possible backward compatibility issues and deprecated features.
anthropic debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..a65cc01f9 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.99.0"
}
diff --git .stats.yml .stats.yml
index d92408173..a3c5d081d 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-5fa16b9a02985ae06e41be14946a9c325dc672fb014b3c19abca65880c6990e6.yml
+openapi_spec_hash: da3e669f65130043b1170048c0727890
+config_hash: d8d5fda350f6db77c784f35429741a2e
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..2a552b8ed 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,53 @@
# Changelog
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git api.md api.md
index 49e6548a8..cad696e7e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -669,10 +709,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..2f29927c6 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.99.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..0d756ef85 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.99.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..61e58a8e8 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..08c898ace 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -104,88 +105,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -195,9 +115,9 @@ export interface EvalStoredCompletionsDataSourceConfig {
schema: Record<string, unknown>;
/**
- * The type of data source. Always `stored_completions`.
+ * The type of data source. Always `stored-completions`.
*/
- type: 'stored_completions';
+ type: 'stored-completions';
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -210,83 +130,6 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
@@ -309,7 +152,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +181,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +243,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -481,7 +273,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +302,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +364,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -653,7 +394,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +423,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +485,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -825,7 +515,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +544,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +606,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -987,15 +626,15 @@ export interface EvalCreateParams {
/**
* The configuration for the data source used for the evaluation runs.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
* A list of graders for all eval runs in this group.
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +683,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +698,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ *, Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored-completions`.
+ */
+ type: 'stored-completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1154,29 +807,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +829,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +875,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..31883e6b5 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -356,7 +356,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +423,28 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +486,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -511,8 +505,8 @@ export namespace RunCreateResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +533,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -749,7 +748,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +815,28 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +878,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -904,8 +897,8 @@ export namespace RunRetrieveResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +925,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1139,10 +1137,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1204,28 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1267,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1297,8 +1286,8 @@ export namespace RunListResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1314,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1543,7 +1537,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1604,28 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1667,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1698,8 +1686,8 @@ export namespace RunCancelResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1714,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1955,9 +1948,9 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
input_messages?:
| CreateEvalResponsesRunDataSource.Template
@@ -2013,12 +2006,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2038,8 +2025,8 @@ export namespace RunCreateParams {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2053,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..1440e865e 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -61,6 +61,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +98,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +125,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +180,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +231,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +280,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1603,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1678,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1695,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1786,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1814,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1846,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1867,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2147,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2167,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2766,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-s,earch).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2822,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2833,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2852,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2907,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..c7ee5a162 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.99.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This PR upgrades the Node.js version used in CI workflows from 18 to 20 and updates the OpenAI Node.js SDK to version 4.99.0 from 4.96.2. The update includes new API features like the reinforcement fine-tuning method, responses x eval API, and various other API enhancements and bug fixes.
Changes
Changes
CI Workflows
- Updated Node.js version from 18 to 20 in CI workflows and publishing workflows
Documentation
- Fixed typos in the README and examples, particularly changing "procesing" to "processing" in error handler examples
- Added extensive JSDoc examples to API methods to improve developer experience
- Marked deprecated API methods with proper
@deprecatedtags - Fixed code formatting in examples
API Enhancements
-
Fine-Tuning
- Added reinforcement fine-tuning API support
- Added new methods for fine-tuning jobs:
pause()andresume() - Restructured the fine-tuning methods and hyperparameters into dedicated types
- Added new alpha graders API for evaluating fine-tuned models
-
Responses and Evals API
- Updated Assistants and Evals API schemas
- Added support for encrypted reasoning content
- Refactored graders into dedicated namespace
- Added support for web search tool enhancements
-
Audio
- Added chunking strategy support for transcriptions
- Updated speech parameters documentation
-
Images
- Added support for transparent backgrounds
- Added additional image sizes
Version Updates
- Updated version from 4.96.2 to 4.99.0 in:
- package.json
- .release-please-manifest.json
- jsr.json
- version.ts
API Spec Stats
- Updated configured endpoints from 97 to 101
- Updated OpenAPI spec hash and URL
sequenceDiagram
participant Dev as Developer
participant CI as CI Workflow
participant NPM as NPM Registry
participant JSR as JSR Registry
Dev->>CI: Push changes
CI->>CI: Bootstrap (with Node 20)
CI->>CI: Run linting
CI->>CI: Run tests
CI->>CI: Check dependencies
alt On release trigger
Dev->>CI: Trigger release
CI->>CI: Setup Node 20
CI->>CI: Install dependencies
CI->>NPM: Publish to NPM
CI->>JSR: Publish to JSR
end
bedrock debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..a65cc01f9 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.99.0"
}
diff --git .stats.yml .stats.yml
index d92408173..a3c5d081d 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-5fa16b9a02985ae06e41be14946a9c325dc672fb014b3c19abca65880c6990e6.yml
+openapi_spec_hash: da3e669f65130043b1170048c0727890
+config_hash: d8d5fda350f6db77c784f35429741a2e
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..2a552b8ed 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,53 @@
# Changelog
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git api.md api.md
index 49e6548a8..cad696e7e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -669,10 +709,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..2f29927c6 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.99.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..0d756ef85 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.99.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..61e58a8e8 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..08c898ace 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -104,88 +105,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -195,9 +115,9 @@ export interface EvalStoredCompletionsDataSourceConfig {
schema: Record<string, unknown>;
/**
- * The type of data source. Always `stored_completions`.
+ * The type of data source. Always `stored-completions`.
*/
- type: 'stored_completions';
+ type: 'stored-completions';
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -210,83 +130,6 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
@@ -309,7 +152,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +181,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +243,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -481,7 +273,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +302,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +364,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -653,7 +394,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +423,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +485,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -825,7 +515,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +544,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +606,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -987,15 +626,15 @@ export interface EvalCreateParams {
/**
* The configuration for the data source used for the evaluation runs.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
* A list of graders for all eval runs in this group.
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +683,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +698,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ *, Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored-completions`.
+ */
+ type: 'stored-completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1154,29 +807,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +829,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +875,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..31883e6b5 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -356,7 +356,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +423,28 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +486,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -511,8 +505,8 @@ export namespace RunCreateResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +533,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -749,7 +748,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +815,28 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +878,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -904,8 +897,8 @@ export namespace RunRetrieveResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +925,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1139,10 +1137,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1204,28 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1267,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1297,8 +1286,8 @@ export namespace RunListResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1314,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1543,7 +1537,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1604,28 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
* A EvalResponsesSource object describing a run data source configuration.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1667,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1698,8 +1686,8 @@ export namespace RunCancelResponse {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1714,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1955,9 +1948,9 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
input_messages?:
| CreateEvalResponsesRunDataSource.Template
@@ -2013,12 +2006,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2038,8 +2025,8 @@ export namespace RunCreateParams {
has_tool_calls?: boolean | null;
/**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2053,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..1440e865e 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -61,6 +61,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +98,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +125,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +180,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +231,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +280,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1603,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1678,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1695,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1786,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1814,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1846,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1867,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2147,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2167,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2766,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-s,earch).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2822,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2833,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2852,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2907,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..c7ee5a162 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.99.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This PR updates the OpenAI Node.js SDK from version 4.96.2 to 4.99.0. The update includes several API enhancements, bug fixes, documentation improvements, and the integration of new features such as reinforcement fine-tuning support, a new graders system, and responses x eval API. The PR also updates the Node.js version used in CI workflows from 18 to 20.
Changes
Changes
Version Upgrade and Package Changes
- Updated version from 4.96.2 to 4.99.0 in multiple files including
package.json,jsr.json, and.release-please-manifest.json - Added changelog entries for versions 4.97.0, 4.98.0, and 4.99.0 in
CHANGELOG.md - Updated API stats in
.stats.yml(configured endpoints increased from 97 to 101)
API Enhancements
-
Fine-Tuning:
- Added reinforcement fine-tuning API support
- Added new methods (
pauseandresume) to the fine-tuning jobs API - Created new fine-tuning methods module with specialized hyperparameters types
- Added alpha graders API for fine-tuning evaluation
-
Graders System:
- Added new Graders resource with various grader models:
- LabelModelGrader
- MultiGrader
- PythonGrader
- ScoreModelGrader
- StringCheckGrader
- TextSimilarityGrader
- Refactored existing grader implementations to use the new system
- Added new Graders resource with various grader models:
-
Responses and Evals API:
- Updated Assistants and Evals API schemas
- Added encrypted reasoning content support
- Improved WebSearchTool with user location capabilities
- Enhanced image generation with more size options and background transparency features
Documentation
- Added extensive JSDoc examples throughout the codebase
- Fixed typo in README.md ("Error codes are as followed:" to "Error codes are as follows:")
- Fixed "procesing" to "processing" in realtime examples
- Added code examples for API methods to improve developer experience
- Added deprecation warnings to functions being replaced
CI Changes
- Updated Node.js version from 18 to 20 in all CI workflow files
Minor Changes
- Improved formatting in examples/tsconfig.json
- Various code structure improvements and refactorings
sequenceDiagram
participant Developer
participant CI as CI/CD Pipeline
participant SDK as OpenAI Node SDK
participant OpenAI as OpenAI API
Developer->>SDK: Submit SDK Enhancement PR
Note over SDK: Update from v4.96.2 to v4.99.0
Developer->>CI: Update Node.js version from 18 to 20
SDK->>SDK: Add new API features
Note over SDK: Reinforcement fine-tuning
Note over SDK: Graders system
Note over SDK: Responses x eval API
SDK->>SDK: Enhance documentation
Note over SDK: Add JSDoc examples
Note over SDK: Add code samples
Note over SDK: Fix typos
CI->>SDK: Run tests on Node.js 20
SDK->>OpenAI: Integrate new API endpoints
Note over OpenAI: Support for 101 endpoints (up from 97)
OpenAI->>SDK: New capabilities available
Note over SDK: Pause/resume fine-tuning jobs
Note over SDK: Encrypted reasoning content
Note over SDK: Enhanced image generation options
SDK->>Developer: PR ready for review
openai debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..989bed91e 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.100.0"
}
diff --git .stats.yml .stats.yml
index d92408173..afa33d93b 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-262e171d0a8150ea1192474d16ba3afdf9a054b399f1a49a9c9b697a3073c136.yml
+openapi_spec_hash: 33e00a48df8f94c94f46290c489f132b
+config_hash: d8d5fda350f6db77c784f35429741a2e
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..adda41e52 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,66 @@
# Changelog
+## 4.100.0 (2025-05-16)
+
+Full Changelog: [v4.99.0...v4.100.0](https://github.com/openai/openai-node/compare/v4.99.0...v4.100.0)
+
+### Features
+
+* **api:** further updates for evals API ([3f6f248](https://github.com/openai/openai-node/commit/3f6f248191b45015924be76fd5154d149c4ed8a0))
+
+
+### Chores
+
+* **internal:** version bump ([5123fe0](https://github.com/openai/openai-node/commit/5123fe08a56f3d0040b1cc67129382f3eacc3cca))
+
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git api.md api.md
index 49e6548a8..cad696e7e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -669,10 +709,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..3c2d41b0f 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.100.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..23205e569 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.100.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/realtime.ts src/resources/beta/realtime/realtime.ts
index 5012b1edd..26fba883e 100644
--- src/resources/beta/realtime/realtime.ts
+++ src/resources/beta/realtime/realtime.ts
@@ -2571,7 +2571,7 @@ export interface TranscriptionSessionUpdatedEvent {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
session: TranscriptionSessionsAPI.TranscriptionSession;
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..83e8c47ad 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
@@ -29,7 +35,7 @@ export class TranscriptionSessions extends APIResource {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
export interface TranscriptionSession {
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..05a656619 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -27,7 +28,8 @@ export class Evals extends APIResource {
/**
* Create the structure of an evaluation that can be used to test a model's
- * performance. An evaluation is a set of testing criteria and a datasource. After
+ * performance. An evaluation is a set of testing criteria and the config for a
+ * data source, which dictates the schema of the data used in the evaluation. After
* creating an evaluation, you can run it on different models and model parameters.
* We support several types of graders and datasources. For more information, see
* the [Evals guide](https://platform.openai.com/docs/guides/evals).
@@ -104,88 +106,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -210,90 +131,13 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalCreateResponse {
/**
@@ -309,7 +153,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +182,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +244,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -465,7 +258,7 @@ export namespace EvalCreateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalRetrieveResponse {
/**
@@ -481,7 +274,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +303,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +365,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -637,7 +379,7 @@ export namespace EvalRetrieveResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalUpdateResponse {
/**
@@ -653,7 +395,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +424,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +486,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -809,7 +500,7 @@ export namespace EvalUpdateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalListResponse {
/**
@@ -825,7 +516,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +545,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +607,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /,**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -985,17 +625,21 @@ export interface EvalDeleteResponse {
export interface EvalCreateParams {
/**
- * The configuration for the data source used for the evaluation runs.
+ * The configuration for the data source used for the evaluation runs. Dictates the
+ * schema of the data used in the evaluation.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
- * A list of graders for all eval runs in this group.
+ * A list of graders for all eval runs in this group. Graders can reference
+ * variables in the data source using double curly braces notation, like
+ * `{{item.variable_name}}`. To reference the model's output, use the `sample`
+ * namespace (ie, `{{sample.output_text}}`).
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +688,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +703,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored_completions`.
+ */
+ type: 'stored_completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1067,7 +725,7 @@ export namespace EvalCreateParams {
export interface LabelModel {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
input: Array<LabelModel.SimpleInputMessage | LabelModel.EvalItem>;
@@ -1154,29 +812,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +834,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +880,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..e761e2160 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -19,7 +19,9 @@ export class Runs extends APIResource {
outputItems: OutputItemsAPI.OutputItems = new OutputItemsAPI.OutputItems(this._client);
/**
- * Create a new evaluation run. This is the endpoint that will kick off grading.
+ * Kicks off a new run for a given evaluation, specifying the data source, and what
+ * model configuration to use to test. The datasource will be validated against the
+ * schema specified in the config of the evaluation.
*/
create(
evalId: string,
@@ -85,7 +87,7 @@ export class RunListResponsesPage extends CursorPage<RunListResponse> {}
*/
export interface CreateEvalCompletionsRunDataSource {
/**
- * A StoredCompletionsRunDataSource configuration describing a set of filters
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalCompletionsRunDataSource.FileContent
@@ -97,6 +99,12 @@ export interface CreateEvalCompletionsRunDataSource {
*/
type: 'completions';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalCompletionsRunDataSource.Template
| CreateEvalCompletionsRunDataSource.ItemReference;
@@ -185,7 +193,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<ResponsesAPI.EasyInputMessage | Template.Message>;
@@ -241,7 +249,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"
*/
item_reference: string;
@@ -279,6 +287,9 @@ export namespace CreateEvalCompletionsRunDataSource {
* eval
*/
export interface CreateEvalJSONLRunDataSource {
+ /**
+ * Determines what populates the `item` namespace in the data source.
+ */
source: CreateEvalJSONLRunDataSource.FileContent | CreateEvalJSONLRunDataSource.FileID;
/**
@@ -356,7 +367,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +434,34 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +503,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -505,14 +516,8 @@ export namespace RunCreateResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +544,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -553,7 +563,7 @@ export namespace RunCreateResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -621,7 +631,7 @@ export namespace RunCreateResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -749,7 +759,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +826,34 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +895,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -898,14 +908,8 @@ export namespace RunRetrieveResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +936,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -946,7 +955,7 @@ export namespace RunRetrieveResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1014,7 +1023,7 @@ export namespace RunRetrieveResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1139,10 +1148,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1215,34 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1284,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1291,14 +1297,8 @@ export namespace RunListResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1325,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1339,7 +1344,7 @@ export namespace RunListResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1407,7 +1412,7 @@ export namespace RunListResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1543,7 +1548,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1615,34 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1684,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1692,14 +1697,8 @@ export namespace RunCancelResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1725,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1740,7 +1744,7 @@ export namespace RunCancelResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1808,7 +1812,7 @@ export namespace RunCancelResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1947,7 +1951,7 @@ export namespace RunCreateParams {
*/
export interface CreateEvalResponsesRunDataSource {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalResponsesRunDataSource.FileContent
@@ -1955,10 +1959,16 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalResponsesRunDataSource.Template
| CreateEvalResponsesRunDataSource.ItemReference;
@@ -2013,12 +2023,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2032,14 +2036,8 @@ export namespace RunCreateParams {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2064,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -2080,7 +2083,7 @@ export namespace RunCreateParams {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -2148,7 +2151,7 @@ export namespace RunCreateParams {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as Lab,elModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..1440e865e 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -61,6 +61,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +98,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +125,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +180,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +231,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +280,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1603,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1678,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1695,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1786,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1814,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1846,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1867,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2147,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2167,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2766,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2822,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2833,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2852,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2907,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..62b43ffce 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.100.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This pull request updates and enhances various parts of the OpenAI Node.js client library. The main goals of this pull request are to:
- Update the Node.js version in CI configurations to version 20.
- Incorporate new features, corrections, and functions as observed in the updates to the changelog.
- Add and update documentation and examples to align with new API endpoints and behavior.
- Introduce new API resources such as
Alpha,Graders, and new grading methods. - Update several dependencies and configuration files.
Possible Issues
- Backward Compatibility: The upgrading of Node version to 20 may cause compatibility issues for users using older versions of Node.
- Deprecation Notices: Existing users need to be made aware of the deprecation changes.
Security Hotspots
No major security hotspots were evident in the provided changes. The updates primarily focused on functionality and documentation.
Changes
Changes
.github/workflows/ci.yml
- Updated Node version from 18 to 20.
.github/workflows/publish-jsr.yml
- Updated Node version from 18 to 20.
.github/workflows/publish-npm.yml
- Updated Node version from 18 to 20.
.release-please-manifest.json
- Bumped version to 4.100.0.
.stats.yml
- Updated configured_endpoints, openapi_spec_url, openapi_spec_hash, and config_hash values.
CHANGELOG.md
- Documented changes for version 4.100.0 and other recent versions.
README.md & Other Documentation Files
- Corrected grammatical errors and updated examples based on new functionality.
src/index.ts
- Exported new modules and resources.
src/resources
- Multiple updates and additions to handle new Alpha and Grader functions.
- Various method documentation updates.
tests
- Added and updated tests corresponding to new and updated functionalities.
Please ensure to run all tests and review the changelog for any breaking changes before merging the pull request.```mermaid sequenceDiagram participant User as User participant Codebase as OpenAI Node Codebase participant Action as Github Action participant Node20 as Node.js 20 participant Linter as Linter participant Tester as Tester
User->>+Codebase: Submit Pull Request
Codebase->>+Action: Trigger CI Workflow
Action->>+Node20: Setup Node.js 20
Node20-->>Action: Node.js 20 Environment
Action->>+Linter: Run Linter
Linter-->>Action: Linting Passed
Action->>+Tester: Run Tests
Tester-->>Action: Tests Passed
Action-->>User: Provide CI Results
</details>
<!-- Generated by gpt-4o-2024-05-13 -->
anthropic debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..989bed91e 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.100.0"
}
diff --git .stats.yml .stats.yml
index d92408173..afa33d93b 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-262e171d0a8150ea1192474d16ba3afdf9a054b399f1a49a9c9b697a3073c136.yml
+openapi_spec_hash: 33e00a48df8f94c94f46290c489f132b
+config_hash: d8d5fda350f6db77c784f35429741a2e
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..adda41e52 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,66 @@
# Changelog
+## 4.100.0 (2025-05-16)
+
+Full Changelog: [v4.99.0...v4.100.0](https://github.com/openai/openai-node/compare/v4.99.0...v4.100.0)
+
+### Features
+
+* **api:** further updates for evals API ([3f6f248](https://github.com/openai/openai-node/commit/3f6f248191b45015924be76fd5154d149c4ed8a0))
+
+
+### Chores
+
+* **internal:** version bump ([5123fe0](https://github.com/openai/openai-node/commit/5123fe08a56f3d0040b1cc67129382f3eacc3cca))
+
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git api.md api.md
index 49e6548a8..cad696e7e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -669,10 +709,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..3c2d41b0f 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.100.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..23205e569 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.100.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/realtime.ts src/resources/beta/realtime/realtime.ts
index 5012b1edd..26fba883e 100644
--- src/resources/beta/realtime/realtime.ts
+++ src/resources/beta/realtime/realtime.ts
@@ -2571,7 +2571,7 @@ export interface TranscriptionSessionUpdatedEvent {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
session: TranscriptionSessionsAPI.TranscriptionSession;
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..83e8c47ad 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
@@ -29,7 +35,7 @@ export class TranscriptionSessions extends APIResource {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
export interface TranscriptionSession {
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..05a656619 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -27,7 +28,8 @@ export class Evals extends APIResource {
/**
* Create the structure of an evaluation that can be used to test a model's
- * performance. An evaluation is a set of testing criteria and a datasource. After
+ * performance. An evaluation is a set of testing criteria and the config for a
+ * data source, which dictates the schema of the data used in the evaluation. After
* creating an evaluation, you can run it on different models and model parameters.
* We support several types of graders and datasources. For more information, see
* the [Evals guide](https://platform.openai.com/docs/guides/evals).
@@ -104,88 +106,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -210,90 +131,13 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalCreateResponse {
/**
@@ -309,7 +153,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +182,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +244,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -465,7 +258,7 @@ export namespace EvalCreateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalRetrieveResponse {
/**
@@ -481,7 +274,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +303,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +365,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -637,7 +379,7 @@ export namespace EvalRetrieveResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalUpdateResponse {
/**
@@ -653,7 +395,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +424,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +486,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -809,7 +500,7 @@ export namespace EvalUpdateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalListResponse {
/**
@@ -825,7 +516,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +545,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +607,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /,**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -985,17 +625,21 @@ export interface EvalDeleteResponse {
export interface EvalCreateParams {
/**
- * The configuration for the data source used for the evaluation runs.
+ * The configuration for the data source used for the evaluation runs. Dictates the
+ * schema of the data used in the evaluation.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
- * A list of graders for all eval runs in this group.
+ * A list of graders for all eval runs in this group. Graders can reference
+ * variables in the data source using double curly braces notation, like
+ * `{{item.variable_name}}`. To reference the model's output, use the `sample`
+ * namespace (ie, `{{sample.output_text}}`).
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +688,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +703,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored_completions`.
+ */
+ type: 'stored_completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1067,7 +725,7 @@ export namespace EvalCreateParams {
export interface LabelModel {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
input: Array<LabelModel.SimpleInputMessage | LabelModel.EvalItem>;
@@ -1154,29 +812,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +834,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +880,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..e761e2160 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -19,7 +19,9 @@ export class Runs extends APIResource {
outputItems: OutputItemsAPI.OutputItems = new OutputItemsAPI.OutputItems(this._client);
/**
- * Create a new evaluation run. This is the endpoint that will kick off grading.
+ * Kicks off a new run for a given evaluation, specifying the data source, and what
+ * model configuration to use to test. The datasource will be validated against the
+ * schema specified in the config of the evaluation.
*/
create(
evalId: string,
@@ -85,7 +87,7 @@ export class RunListResponsesPage extends CursorPage<RunListResponse> {}
*/
export interface CreateEvalCompletionsRunDataSource {
/**
- * A StoredCompletionsRunDataSource configuration describing a set of filters
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalCompletionsRunDataSource.FileContent
@@ -97,6 +99,12 @@ export interface CreateEvalCompletionsRunDataSource {
*/
type: 'completions';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalCompletionsRunDataSource.Template
| CreateEvalCompletionsRunDataSource.ItemReference;
@@ -185,7 +193,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<ResponsesAPI.EasyInputMessage | Template.Message>;
@@ -241,7 +249,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"
*/
item_reference: string;
@@ -279,6 +287,9 @@ export namespace CreateEvalCompletionsRunDataSource {
* eval
*/
export interface CreateEvalJSONLRunDataSource {
+ /**
+ * Determines what populates the `item` namespace in the data source.
+ */
source: CreateEvalJSONLRunDataSource.FileContent | CreateEvalJSONLRunDataSource.FileID;
/**
@@ -356,7 +367,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +434,34 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +503,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -505,14 +516,8 @@ export namespace RunCreateResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +544,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -553,7 +563,7 @@ export namespace RunCreateResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -621,7 +631,7 @@ export namespace RunCreateResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -749,7 +759,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +826,34 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +895,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -898,14 +908,8 @@ export namespace RunRetrieveResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +936,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -946,7 +955,7 @@ export namespace RunRetrieveResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1014,7 +1023,7 @@ export namespace RunRetrieveResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1139,10 +1148,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1215,34 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1284,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1291,14 +1297,8 @@ export namespace RunListResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1325,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1339,7 +1344,7 @@ export namespace RunListResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1407,7 +1412,7 @@ export namespace RunListResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1543,7 +1548,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1615,34 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1684,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1692,14 +1697,8 @@ export namespace RunCancelResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1725,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1740,7 +1744,7 @@ export namespace RunCancelResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1808,7 +1812,7 @@ export namespace RunCancelResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1947,7 +1951,7 @@ export namespace RunCreateParams {
*/
export interface CreateEvalResponsesRunDataSource {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalResponsesRunDataSource.FileContent
@@ -1955,10 +1959,16 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalResponsesRunDataSource.Template
| CreateEvalResponsesRunDataSource.ItemReference;
@@ -2013,12 +2023,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2032,14 +2036,8 @@ export namespace RunCreateParams {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2064,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -2080,7 +2083,7 @@ export namespace RunCreateParams {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -2148,7 +2151,7 @@ export namespace RunCreateParams {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as Lab,elModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..1440e865e 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -61,6 +61,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +98,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +125,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +180,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +231,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +280,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1603,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1678,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1695,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1786,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1814,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1846,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1867,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2147,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2167,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2766,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2822,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2833,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2852,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2907,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..62b43ffce 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.100.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This PR updates the OpenAI Node.js SDK to version 4.100.0, which includes new API features, bug fixes, and documentation improvements. The primary changes include upgrading the Node.js version in CI workflows from 18 to 20, adding support for new endpoints and models, enhancing existing API endpoints with additional parameters, and improving the documentation with examples.
Changes
Changes
CI and Build Configuration
- Updated Node.js version from 18 to 20 in CI workflows (ci.yml, publish-jsr.yml, publish-npm.yml)
- Bumped version from 4.96.2 to 4.100.0 in package.json, jsr.json, and .release-please-manifest.json
- Updated .stats.yml with new endpoint count and API spec URLs
API Enhancements
-
Fine-Tuning:
- Added support for reinforcement fine-tuning
- Added new
Methodsclass with support for Supervised, DPO and Reinforcement methods - Added pause/resume functionality for fine-tuning jobs
- Added alpha graders endpoint for fine-tuning validation and running
-
Graders:
- Added a new
Gradersresource with various grader types (LabelModel, MultiGrader, Python, etc.) - Moved grader models from Evals to a dedicated module
- Added a new
-
Evals API:
- Updated with improved integration with the grader models
- Added support for new data source configurations
- Deprecated some older components in favor of new system
-
Image Generation:
- Added support for new image sizes
- Added background transparency options
-
Audio Transcription:
- Added chunking strategy options (auto, VAD config, etc.)
Documentation
- Fixed typos (e.g., "procesing" → "processing" in realtime examples)
- Added code examples to methods using JSDoc comments
- Fixed "Error codes are as followed" → "Error codes are as follows" in README.md
- Updated API documentation in api.md
Other Changes
- Enhanced type definitions across the SDK
- Added deprecation warnings to outdated methods/parameters
- Fixed formatting in examples/tsconfig.json
sequenceDiagram
participant Dev as Developer
participant SDK as OpenAI Node SDK
participant API as OpenAI API
Dev->>SDK: Update to version 4.100.0
Note over SDK: Node.js 20 support
Dev->>SDK: Use new reinforcement fine-tuning
SDK->>API: Configure with method.reinforcement
API-->>SDK: Fine-tuning job created
SDK-->>Dev: Returns job details
Dev->>SDK: Pause running fine-tuning job
SDK->>API: POST /fine_tuning/jobs/{id}/pause
API-->>SDK: Job paused response
SDK-->>Dev: Returns updated job status
Dev->>SDK: Resume paused fine-tuning job
SDK->>API: POST /fine_tuning/jobs/{id}/resume
API-->>SDK: Job resumed response
SDK-->>Dev: Returns updated job status
Dev->>SDK: Create image with transparency
SDK->>API: POST /images/edits with background=transparent
API-->>SDK: Returns created image
SDK-->>Dev: Returns image data
Dev->>SDK: Use transcription with auto chunking
SDK->>API: POST /audio/transcriptions with chunking_strategy
API-->>SDK: Returns transcribed text
SDK-->>Dev: Returns transcription data
Dev->>SDK: Use grader validation
SDK->>API: POST /fine_tuning/alpha/graders/validate
API-->>SDK: Returns validation results
SDK-->>Dev: Returns grader validation data
bedrock debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..989bed91e 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.100.0"
}
diff --git .stats.yml .stats.yml
index d92408173..afa33d93b 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-262e171d0a8150ea1192474d16ba3afdf9a054b399f1a49a9c9b697a3073c136.yml
+openapi_spec_hash: 33e00a48df8f94c94f46290c489f132b
+config_hash: d8d5fda350f6db77c784f35429741a2e
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..adda41e52 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,66 @@
# Changelog
+## 4.100.0 (2025-05-16)
+
+Full Changelog: [v4.99.0...v4.100.0](https://github.com/openai/openai-node/compare/v4.99.0...v4.100.0)
+
+### Features
+
+* **api:** further updates for evals API ([3f6f248](https://github.com/openai/openai-node/commit/3f6f248191b45015924be76fd5154d149c4ed8a0))
+
+
+### Chores
+
+* **internal:** version bump ([5123fe0](https://github.com/openai/openai-node/commit/5123fe08a56f3d0040b1cc67129382f3eacc3cca))
+
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git api.md api.md
index 49e6548a8..cad696e7e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -669,10 +709,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..3c2d41b0f 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.100.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..23205e569 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.100.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/realtime.ts src/resources/beta/realtime/realtime.ts
index 5012b1edd..26fba883e 100644
--- src/resources/beta/realtime/realtime.ts
+++ src/resources/beta/realtime/realtime.ts
@@ -2571,7 +2571,7 @@ export interface TranscriptionSessionUpdatedEvent {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
session: TranscriptionSessionsAPI.TranscriptionSession;
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..83e8c47ad 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
@@ -29,7 +35,7 @@ export class TranscriptionSessions extends APIResource {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
export interface TranscriptionSession {
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..05a656619 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -27,7 +28,8 @@ export class Evals extends APIResource {
/**
* Create the structure of an evaluation that can be used to test a model's
- * performance. An evaluation is a set of testing criteria and a datasource. After
+ * performance. An evaluation is a set of testing criteria and the config for a
+ * data source, which dictates the schema of the data used in the evaluation. After
* creating an evaluation, you can run it on different models and model parameters.
* We support several types of graders and datasources. For more information, see
* the [Evals guide](https://platform.openai.com/docs/guides/evals).
@@ -104,88 +106,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -210,90 +131,13 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalCreateResponse {
/**
@@ -309,7 +153,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +182,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +244,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -465,7 +258,7 @@ export namespace EvalCreateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalRetrieveResponse {
/**
@@ -481,7 +274,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +303,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +365,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -637,7 +379,7 @@ export namespace EvalRetrieveResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalUpdateResponse {
/**
@@ -653,7 +395,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +424,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +486,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -809,7 +500,7 @@ export namespace EvalUpdateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalListResponse {
/**
@@ -825,7 +516,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +545,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +607,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /,**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -985,17 +625,21 @@ export interface EvalDeleteResponse {
export interface EvalCreateParams {
/**
- * The configuration for the data source used for the evaluation runs.
+ * The configuration for the data source used for the evaluation runs. Dictates the
+ * schema of the data used in the evaluation.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
- * A list of graders for all eval runs in this group.
+ * A list of graders for all eval runs in this group. Graders can reference
+ * variables in the data source using double curly braces notation, like
+ * `{{item.variable_name}}`. To reference the model's output, use the `sample`
+ * namespace (ie, `{{sample.output_text}}`).
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +688,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +703,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored_completions`.
+ */
+ type: 'stored_completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1067,7 +725,7 @@ export namespace EvalCreateParams {
export interface LabelModel {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
input: Array<LabelModel.SimpleInputMessage | LabelModel.EvalItem>;
@@ -1154,29 +812,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +834,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +880,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..e761e2160 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -19,7 +19,9 @@ export class Runs extends APIResource {
outputItems: OutputItemsAPI.OutputItems = new OutputItemsAPI.OutputItems(this._client);
/**
- * Create a new evaluation run. This is the endpoint that will kick off grading.
+ * Kicks off a new run for a given evaluation, specifying the data source, and what
+ * model configuration to use to test. The datasource will be validated against the
+ * schema specified in the config of the evaluation.
*/
create(
evalId: string,
@@ -85,7 +87,7 @@ export class RunListResponsesPage extends CursorPage<RunListResponse> {}
*/
export interface CreateEvalCompletionsRunDataSource {
/**
- * A StoredCompletionsRunDataSource configuration describing a set of filters
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalCompletionsRunDataSource.FileContent
@@ -97,6 +99,12 @@ export interface CreateEvalCompletionsRunDataSource {
*/
type: 'completions';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalCompletionsRunDataSource.Template
| CreateEvalCompletionsRunDataSource.ItemReference;
@@ -185,7 +193,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<ResponsesAPI.EasyInputMessage | Template.Message>;
@@ -241,7 +249,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"
*/
item_reference: string;
@@ -279,6 +287,9 @@ export namespace CreateEvalCompletionsRunDataSource {
* eval
*/
export interface CreateEvalJSONLRunDataSource {
+ /**
+ * Determines what populates the `item` namespace in the data source.
+ */
source: CreateEvalJSONLRunDataSource.FileContent | CreateEvalJSONLRunDataSource.FileID;
/**
@@ -356,7 +367,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +434,34 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +503,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -505,14 +516,8 @@ export namespace RunCreateResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +544,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -553,7 +563,7 @@ export namespace RunCreateResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -621,7 +631,7 @@ export namespace RunCreateResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -749,7 +759,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +826,34 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +895,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -898,14 +908,8 @@ export namespace RunRetrieveResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +936,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -946,7 +955,7 @@ export namespace RunRetrieveResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1014,7 +1023,7 @@ export namespace RunRetrieveResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1139,10 +1148,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1215,34 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1284,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1291,14 +1297,8 @@ export namespace RunListResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1325,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1339,7 +1344,7 @@ export namespace RunListResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1407,7 +1412,7 @@ export namespace RunListResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1543,7 +1548,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1615,34 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1684,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1692,14 +1697,8 @@ export namespace RunCancelResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1725,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1740,7 +1744,7 @@ export namespace RunCancelResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1808,7 +1812,7 @@ export namespace RunCancelResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1947,7 +1951,7 @@ export namespace RunCreateParams {
*/
export interface CreateEvalResponsesRunDataSource {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalResponsesRunDataSource.FileContent
@@ -1955,10 +1959,16 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalResponsesRunDataSource.Template
| CreateEvalResponsesRunDataSource.ItemReference;
@@ -2013,12 +2023,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2032,14 +2036,8 @@ export namespace RunCreateParams {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2064,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -2080,7 +2083,7 @@ export namespace RunCreateParams {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -2148,7 +2151,7 @@ export namespace RunCreateParams {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as Lab,elModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..1440e865e 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -61,6 +61,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +98,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +125,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +180,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +231,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +280,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -1581,11 +1603,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1678,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1695,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1758,19 +1786,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1814,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,13 +1846,13 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
@@ -1839,7 +1867,7 @@ export namespace ResponseInputItem {
/**
* The type of item to reference. Always `item_reference`.
*/
- type: 'item_reference';
+ type?: 'item_reference' | null;
}
}
@@ -2119,7 +2147,9 @@ export namespace ResponseOutputText {
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +2167,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2730,11 +2766,9 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool = FileSearchTool | FunctionTool | WebSearchTool | ComputerTool;
/**
* Use this option to force the model to call a specific function.
@@ -2788,10 +2822,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +2833,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +2852,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2869,6 +2907,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..62b43ffce 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.100.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
Description
This PR updates the OpenAI Node.js SDK from version 4.96.2 to 4.100.0, incorporating new API features and improvements. The changes include support for reinforcement fine-tuning, updates to the Evals API, various text fixes, and updates to Node.js runtime requirements from Node 18 to Node 20.
Changes
Changes
GitHub Workflows
- Changed Node.js version from 18 to 20 in CI, publish-jsr, and publish-npm workflows
Version & Metadata Updates
- Updated version from 4.96.2 to 4.100.0 in package.json, jsr.json, and .release-please-manifest.json
- Updated .stats.yml with new endpoint count (97 → 101) and new API spec hashes
CHANGELOG.md
- Added entries for versions 4.97.0 through 4.100.0 including:
- Evals API updates
- Reinforcement fine-tuning API support
- Added example TSDocs
- Various documentation fixes
API Additions & Updates
-
Fine-Tuning
- Added reinforcement fine-tuning API support
- Added fine-tuning alpha graders API
- Added pause and resume methods for fine-tuning jobs
-
Graders
- Added new Graders resource with various grader models (StringCheck, TextSimilarity, Python, ScoreModel, etc.)
-
Evals API
- Refactored grader types to use the new Graders module
- Updated data source configurations
-
Audio API
- Added chunking_strategy parameter to transcriptions
- Added documentation examples
-
Images API
- Added background transparency support for image edit
- Added new image size options
-
Responses API
- Added support for reasoning encryption
- Updated WebSearchTool and other tools
Documentation
- Added JSDoc examples to many API methods
- Fixed typos (e.g., "procesing" → "processing")
- Updated error codes section in README.md
- Fixed formatting in examples/tsconfig.json
sequenceDiagram
participant Dev as Developer
participant SDK as OpenAI SDK
participant API as OpenAI API
Dev->>SDK: Update from v4.96.2 to v4.100.0
Note over SDK: Now using Node 20
alt New Fine-Tuning Features
Dev->>SDK: Create reinforcement fine-tuning job
SDK->>API: POST /fine_tuning/jobs
API-->>SDK: Return job details
Dev->>SDK: Pause fine-tuning job
SDK->>API: POST /fine_tuning/jobs/{id}/pause
API-->>SDK: Return updated job status
Dev->>SDK: Resume fine-tuning job
SDK->>API: POST /fine_tuning/jobs/{id}/resume
API-->>SDK: Return updated job status
end
alt New Graders API
Dev->>SDK: Create grader
SDK->>API: Use grader via fine-tuning
API-->>SDK: Return grading results
end
alt Enhanced API Features
Dev->>SDK: Use audio transcription with chunking_strategy
SDK->>API: POST /audio/transcriptions with chunking params
API-->>SDK: Return chunked transcription
Dev->>SDK: Create image with transparency settings
SDK->>API: POST /images/edits with background: "transparent"
API-->>SDK: Return transparent image
end
anthropic debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..514215a1e 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.101.0"
}
diff --git .stats.yml .stats.yml
index d92408173..4b4f19c91 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-a5651cb97f86d1e2531af6aef8c5230f1ea350560fbae790ca2e481b30a6c217.yml
+openapi_spec_hash: 66a5104fd3bb43383cf919225df7a6fd
+config_hash: bb657c3fed232a56930035de3aaed936
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..1e427d912 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,80 @@
# Changelog
+## 4.101.0 (2025-05-21)
+
+Full Changelog: [v4.100.0...v4.101.0](https://github.com/openai/openai-node/compare/v4.100.0...v4.101.0)
+
+### Features
+
+* **api:** new API tools ([fb4014f](https://github.com/openai/openai-node/commit/fb4014ffac7b220d37bd03c94fa745386b010bf0))
+
+
+### Chores
+
+* **docs:** grammar improvements ([7761cfb](https://github.com/openai/openai-node/commit/7761cfb0a8a56d056a73c046a6a613f66ada4abe))
+* **internal:** version bump ([b40e830](https://github.com/openai/openai-node/commit/b40e8302ec11683b6a360a050236ad1284afc760))
+
+## 4.100.0 (2025-05-16)
+
+Full Changelog: [v4.99.0...v4.100.0](https://github.com/openai/openai-node/compare/v4.99.0...v4.100.0)
+
+### Features
+
+* **api:** further updates for evals API ([3f6f248](https://github.com/openai/openai-node/commit/3f6f248191b45015924be76fd5154d149c4ed8a0))
+
+
+### Chores
+
+* **internal:** version bump ([5123fe0](https://github.com/openai/openai-node/commit/5123fe08a56f3d0040b1cc67129382f3eacc3cca))
+
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git SECURITY.md SECURITY.md
index 3b3bd8a66..4adb0c54f 100644
--- SECURITY.md
+++ SECURITY.md
@@ -16,13 +16,13 @@ before making any information public.
## Reporting Non-SDK Related Security Issues
If you encounter security issues that are not directly related to SDKs but pertain to the services
-or products provided by OpenAI please follow the respective company's security reporting guidelines.
+or products provided by OpenAI, please follow the respective company's security reporting guidelines.
### OpenAI Terms and Policies
Our Security Policy can be found at [Security Policy URL](https://openai.com/policies/coordinated-vulnerability-disclosure-policy).
-Please contact [email protected] for any questions or concerns regarding security of our services.
+Please contact [email protected] for any questions or concerns regarding the security of our services.
---
diff --git api.md api.md
index 49e6548a8..f7fb47e3e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -605,6 +645,10 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionWebSearch</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallGeneratingEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallPartialImageEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncludable</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncompleteEvent</a></code>
@@ -618,6 +662,14 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseInputMessageItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInputText</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDoneEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputAudio</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItemAddedEvent</a></code>
@@ -625,7 +677,13 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputMessage</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputRefusal</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputText</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseOutputTextAnnotationAddedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseQueuedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartAddedEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryTextDeltaEvent</a></code>
@@ -669,10 +727,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..42bf0a235 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.101.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..e8cd7277c 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.101.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/realtime.ts src/resources/beta/realtime/realtime.ts
index 5012b1edd..26fba883e 100644
--- src/resources/beta/realtime/realtime.ts
+++ src/resources/beta/realtime/realtime.ts
@@ -2571,7 +2571,7 @@ export interface TranscriptionSessionUpdatedEvent {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
session: TranscriptionSessionsAPI.TranscriptionSession;
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..83e8c47ad 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
@@ -29,7 +35,7 @@ export class TranscriptionSessions extends APIResource {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
export interface TranscriptionSession {
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..05a656619 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -27,7 +28,8 @@ export class Evals extends APIResource {
/**
* Create the structure of an evaluation that can be used to test a model's
- * performance. An evaluation is a set of testing criteria and a datasource. After
+ * performance. An evaluation is a set of testing criteria and the config for a
+ * data source, which dictates the schema of the data used in the evaluation. After
* creating an evaluation, you can run it on different models and model parameters.
* We support several types of graders and datasources. For more information, see
* the [Evals guide](https://platform.openai.com/docs/guides/evals).
@@ -104,88 +106,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -210,90 +131,13 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalCreateResponse {
/**
@@ -309,7 +153,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +182,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +244,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -465,7 +258,7 @@ export namespace EvalCreateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalRetrieveResponse {
/**
@@ -481,7 +274,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +303,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +365,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -637,7 +379,7 @@ export namespace EvalRetrieveResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalUpdateResponse {
/**
@@ -653,7 +395,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +424,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +486,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
*, The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -809,7 +500,7 @@ export namespace EvalUpdateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalListResponse {
/**
@@ -825,7 +516,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +545,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +607,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -985,17 +625,21 @@ export interface EvalDeleteResponse {
export interface EvalCreateParams {
/**
- * The configuration for the data source used for the evaluation runs.
+ * The configuration for the data source used for the evaluation runs. Dictates the
+ * schema of the data used in the evaluation.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
- * A list of graders for all eval runs in this group.
+ * A list of graders for all eval runs in this group. Graders can reference
+ * variables in the data source using double curly braces notation, like
+ * `{{item.variable_name}}`. To reference the model's output, use the `sample`
+ * namespace (ie, `{{sample.output_text}}`).
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +688,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +703,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored_completions`.
+ */
+ type: 'stored_completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1067,7 +725,7 @@ export namespace EvalCreateParams {
export interface LabelModel {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
input: Array<LabelModel.SimpleInputMessage | LabelModel.EvalItem>;
@@ -1154,29 +812,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +834,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +880,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..e761e2160 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -19,7 +19,9 @@ export class Runs extends APIResource {
outputItems: OutputItemsAPI.OutputItems = new OutputItemsAPI.OutputItems(this._client);
/**
- * Create a new evaluation run. This is the endpoint that will kick off grading.
+ * Kicks off a new run for a given evaluation, specifying the data source, and what
+ * model configuration to use to test. The datasource will be validated against the
+ * schema specified in the config of the evaluation.
*/
create(
evalId: string,
@@ -85,7 +87,7 @@ export class RunListResponsesPage extends CursorPage<RunListResponse> {}
*/
export interface CreateEvalCompletionsRunDataSource {
/**
- * A StoredCompletionsRunDataSource configuration describing a set of filters
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalCompletionsRunDataSource.FileContent
@@ -97,6 +99,12 @@ export interface CreateEvalCompletionsRunDataSource {
*/
type: 'completions';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalCompletionsRunDataSource.Template
| CreateEvalCompletionsRunDataSource.ItemReference;
@@ -185,7 +193,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<ResponsesAPI.EasyInputMessage | Template.Message>;
@@ -241,7 +249,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"
*/
item_reference: string;
@@ -279,6 +287,9 @@ export namespace CreateEvalCompletionsRunDataSource {
* eval
*/
export interface CreateEvalJSONLRunDataSource {
+ /**
+ * Determines what populates the `item` namespace in the data source.
+ */
source: CreateEvalJSONLRunDataSource.FileContent | CreateEvalJSONLRunDataSource.FileID;
/**
@@ -356,7 +367,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +434,34 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +503,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -505,14 +516,8 @@ export namespace RunCreateResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +544,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -553,7 +563,7 @@ export namespace RunCreateResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -621,7 +631,7 @@ export namespace RunCreateResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -749,7 +759,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +826,34 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +895,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -898,14 +908,8 @@ export namespace RunRetrieveResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +936,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -946,7 +955,7 @@ export namespace RunRetrieveResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1014,7 +1023,7 @@ export namespace RunRetrieveResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1139,10 +1148,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1215,34 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1284,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1291,14 +1297,8 @@ export namespace RunListResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1325,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1339,7 +1344,7 @@ export namespace RunListResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1407,7 +1412,7 @@ export namespace RunListResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1543,7 +1548,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1615,34 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1684,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1692,14 +1697,8 @@ export namespace RunCancelResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1725,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1740,7 +1744,7 @@ export namespace RunCancelResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1808,7 +1812,7 @@ export namespace RunCancelResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1947,7 +1951,7 @@ export namespace RunCreateParams {
*/
export interface CreateEvalResponsesRunDataSource {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalResponsesRunDataSource.FileContent
@@ -1955,10 +1959,16 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalResponsesRunDataSource.Template
| CreateEvalResponsesRunDataSource.ItemReference;
@@ -2013,12 +2023,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2032,14 +2036,8 @@ export namespace RunCreateParams {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2064,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -2080,7 +2083,7 @@ export namespace RunCreateParams {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -2148,7 +2151,7 @@ export namespace RunCreateParams {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | Tex,tSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..fdf51f224 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -37,7 +37,13 @@ export type ParsedResponseOutputItem<ParsedT> =
| ResponseFileSearchToolCall
| ResponseFunctionWebSearch
| ResponseComputerToolCall
- | ResponseReasoningItem;
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
export interface ParsedResponse<ParsedT> extends Response {
output: Array<ParsedResponseOutputItem<ParsedT>>;
@@ -61,6 +67,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +104,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +131,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +186,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +237,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +286,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -392,6 +420,12 @@ export interface Response {
*/
top_p: number | null;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* An upper bound for the number of tokens that can be generated for a response,
* including visible output tokens and
@@ -437,7 +471,7 @@ export interface Response {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
status?: ResponseStatus;
@@ -665,6 +699,11 @@ export interface ResponseCodeInterpreterToolCall {
* The type of the code interpreter tool call. Always `code_interpreter_call`.
*/
type: 'code_interpreter_call';
+
+ /**
+ * The ID of the container used to run the code.
+ */
+ container_id?: string;
}
export namespace ResponseCodeInterpreterToolCall {
@@ -1557,6 +1596,114 @@ export interface ResponseFunctionWebSearch {
type: 'web_search_call';
}
+/**
+ * Emitted when an image generation tool call has completed and the final image is
+ * available.
+ */
+export interface ResponseImageGenCallCompletedEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.completed'.
+ */
+ type: 'response.image_generation_call.completed';
+}
+
+/**
+ * Emitted when an image generation tool call is actively generating an image
+ * (intermediate state).
+ */
+export interface ResponseImageGenCallGeneratingEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.generating'.
+ */
+ type: 'response.image_generation_call.generating';
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number?: number;
+}
+
+/**
+ * Emitted when an image generation tool call is in progress.
+ */
+export interface ResponseImageGenCallInProgressEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.in_progress'.
+ */
+ type: 'response.image_generation_call.in_progress';
+}
+
+/**
+ * Emitted when a partial image is available during image generation streaming.
+ */
+export interface ResponseImageGenCallPartialImageEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * Base64-encoded partial image data, suitable for rendering as an image.
+ */
+ partial_image_b64: string;
+
+ /**
+ * 0-based index for the partial image (backend is 1-based, but this is 0-based for
+ * the user).
+ */
+ partial_image_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.partial_image'.
+ */
+ type: 'response.image_generation_call.partial_image';
+}
+
/**
* Emitted when the response is in progress.
*/
@@ -1581,11 +1728,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1803,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1820,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1704,6 +1857,14 @@ export type ResponseInputItem =
| ResponseFunctionToolCall
| ResponseInputItem.FunctionCallOutput
| ResponseReasoningItem
+ | ResponseInputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseInputItem.LocalShellCall
+ | ResponseInputItem.LocalShellCallOutput
+ | ResponseInputItem.McpListTools
+ | ResponseInputItem.McpApprovalRequest
+ | ResponseInputItem.McpApprovalResponse
+ | ResponseInputItem.McpCall
| ResponseInputItem.ItemReference;
export namespace ResponseInputItem {
@@ -1758,19 +1919,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1947,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,172 +1979,1079 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
- * An internal identifier for an item to reference.
+ * An image generation request made by the model.
*/
- export interface ItemReference {
+ export interface ImageGenerationCall {
/**
- * The ID of the item to reference.
+ * The unique ID of the image generation call.
*/
id: string;
/**
- * The type of item to reference. Always `item_reference`.
+ * The generated image encoded in base64.
*/
- type: 'item_reference';
- }
-}
+ result: string | null;
-/**
- * A list of one or many input items to the model, containing different content
- * types.
- */
-export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
-export interface ResponseInputMessageItem {
- /**
- * The unique ID of the message input.
- */
- id: string;
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
/**
- * A list of one or many input items to the model, containing different content
- * types.
+ * A tool call to run a command on the local shell.
*/
- content: ResponseInputMessageContentList;
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
- /**
- * The role of the message input. One of `user`, `system`, or `developer`.
- */
- role: 'user' | 'system' | 'developer';
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
- /**
- * The status of item. One of `in_progress`, `completed`, or `incomplete`.
- * Populated when items are returned via API.
- */
- status?: 'in_progress' | 'completed' | 'incomplete';
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
- /**
- * The type of the message input. Always set to `message`.
- */
- type?: 'message';
-}
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
-/**
- * A text input to the model.
- */
-export interface ResponseInputText {
- /**
- * The text input to the model.
- */
- text: string;
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
- /**
- * The type of the input item. Always `input_text`.
- */
- type: 'input_text';
-}
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
-/**
- * Content item used to generate a response.
- */
-export type ResponseItem =
- | ResponseInputMessageItem
- | ResponseOutputMessage
- | ResponseFileSearchToolCall
- | ResponseComputerToolCall
- | ResponseComputerToolCallOutputItem
- | ResponseFunctionWebSearch
- | ResponseFunctionToolCallItem
- | ResponseFunctionToolCallOutputItem;
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
-/**
- * An audio output from the model.
- */
-export interface ResponseOutputAudio {
- /**
- * Base64-encoded audio data from the model.
- */
- data: string;
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
- /**
- * The transcript of the audio data from the model.
- */
- transcript: string;
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
- /**
- * The type of the output audio. Always `output_audio`.
- */
- type: 'output_audio';
-}
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
-/**
- * An output message from the model.
- */
-export type ResponseOutputItem =
- | ResponseOutputMessage
- | ResponseFileSearchToolCall
- | ResponseFunctionToolCall
- | ResponseFunctionWebSearch
- | ResponseComputerToolCall
- | ResponseReasoningItem;
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
-/**
- * Emitted when a new output item is added.
- */
-export interface ResponseOutputItemAddedEvent {
/**
- * The output item that was added.
+ * The output of a local shell tool call.
*/
- item: ResponseOutputItem;
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
- /**
- * The index of the output item that was added.
- */
- output_index: number;
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
- /**
- * The type of the event. Always `response.output_item.added`.
- */
- type: 'response.output_item.added';
-}
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
-/**
- * Emitted when an output item is marked done.
- */
-export interface ResponseOutputItemDoneEvent {
- /**
- * The output item that was marked done.
- */
- item: ResponseOutputItem;
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
/**
- * The index of the output item that was marked done.
+ * A list of tools available on an MCP server.
*/
- output_index: number;
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
- /**
- * The type of the event. Always `response.output_item.done`.
- */
- type: 'response.output_item.done';
-}
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
-/**
- * An output message from the model.
- */
-export interface ResponseOutputMessage {
- /**
- * The unique ID of the output message.
- */
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * The unique ID of the approval response
+ */
+ id?: string | null;
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * An internal identifier for an item to reference.
+ */
+ export interface ItemReference {
+ /**
+ * The ID of the item to reference.
+ */
+ id: string;
+
+ /**
+ * The type of item to reference. Always `item_reference`.
+ */
+ type?: 'item_reference' | null;
+ }
+}
+
+/**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+
+export interface ResponseInputMessageItem {
+ /**
+ * The unique ID of the message input.
+ */
+ id: string;
+
+ /**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+ content: ResponseInputMessageContentList;
+
+ /**
+ * The role of the message input. One of `user`, `system`, or `developer`.
+ */
+ role: 'user' | 'system' | 'developer';
+
+ /**
+ * The status of item. One of `in_progress`, `completed`, or `incomplete`.
+ * Populated when items are returned via API.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the message input. Always set to `message`.
+ */
+ type?: 'message';
+}
+
+/**
+ * A text input to the model.
+ */
+export interface ResponseInputText {
+ /**
+ * The text input to the model.
+ */
+ text: string;
+
+ /**
+ * The type of the input item. Always `input_text`.
+ */
+ type: 'input_text';
+}
+
+/**
+ * Content item used to generate a response.
+ */
+export type ResponseItem =
+ | ResponseInputMessageItem
+ | ResponseOutputMessage
+ | ResponseFileSearchToolCall
+ | ResponseComputerToolCall
+ | ResponseComputerToolCallOutputItem
+ | ResponseFunctionWebSearch
+ | ResponseFunctionToolCallItem
+ | ResponseFunctionToolCallOutputItem
+ | ResponseItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseItem.LocalShellCall
+ | ResponseItem.LocalShellCallOutput
+ | ResponseItem.McpListTools
+ | ResponseItem.McpApprovalRequest
+ | ResponseItem.McpApprovalResponse
+ | ResponseItem.McpCall;
+
+export namespace ResponseItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * The output of a local shell tool call.
+ */
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
+
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
+
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The unique ID of the approval response
+ */
+ id: string;
+
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the arguments of an MCP tool
+ * call.
+ */
+export interface ResponseMcpCallArgumentsDeltaEvent {
+ /**
+ * The partial update to the arguments for the MCP tool call.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_delta'.
+ */
+ type: 'response.mcp_call.arguments_delta';
+}
+
+/**
+ * Emitted when the arguments for an MCP tool call are finalized.
+ */
+export interface ResponseMcpCallArgumentsDoneEvent {
+ /**
+ * The finalized arguments for the MCP tool call.
+ */
+ arguments: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_done'.
+ */
+ type: 'response.mcp_call.arguments_done';
+}
+
+/**
+ * Emitted when an MCP tool call has completed successfully.
+ */
+export interface ResponseMcpCallCompletedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_call.completed'.
+ */
+ type: 'response.mcp_call.completed';
+}
+
+/**
+ * Emitted when an MCP tool call has failed.
+ */
+export interface ResponseMcpCallFailedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_call.failed'.
+ */
+ type: 'response.mcp_call.failed';
+}
+
+/**
+ * Emitted when an MCP tool call is in progress.
+ */
+export interface ResponseMcpCallInProgressEvent {
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.in_progress'.
+ */
+ type: 'response.mcp_call.in_progress';
+}
+
+/**
+ * Emitted when the list of available MCP tools has been successfully retrieved.
+ */
+export interface ResponseMcpListToolsCompletedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.completed'.
+ */
+ type: 'response.mcp_list_tools.completed';
+}
+
+/**
+ * Emitted when the attempt to list available MCP tools has failed.
+ */
+export interface ResponseMcpListToolsFailedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.failed'.
+ */
+ type: 'response.mcp_list_tools.failed';
+}
+
+/**
+ * Emitted when the system is in the process of retrieving the list of available
+ * MCP tools.
+ */
+export interface ResponseMcpListToolsInProgressEvent {
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.in_progress'.
+ */
+ type: 'response.mcp_list_tools.in_progress';
+}
+
+/**
+ * An audio output from the model.
+ */
+export interface ResponseOutputAudio {
+ /**
+ * Base64-encoded audio data from the model.
+ */
+ data: string;
+
+ /**
+ * The transcript of the audio data from the model.
+ */
+ transcript: string;
+
+ /**
+ * The type of the output audio. Always `output_audio`.
+ */
+ type: 'output_audio';
+}
+
+/**
+ * An output message from the model.
+ */
+export type ResponseOutputItem =
+ | ResponseOutputMessage
+ | ResponseFileSearchToolCall
+ | ResponseFunctionToolCall
+ | ResponseFunctionWebSearch
+ | ResponseComputerToolCall
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
+
+export namespace ResponseOutputItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+}
+
+/**
+ * Emitted when a new output item is added.
+ */
+export interface ResponseOutputItemAddedEvent {
+ /**
+ * The output item that was added.
+ */
+ item: ResponseOutputItem;
+
+ /**
+ * The index of the output item that was added.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always `response.output_item.added`.
+ */
+ type: 'response.output_item.added';
+}
+
+/**
+ * Emitted when an output item is marked done.
+ */
+export interface ResponseOutputItemDoneEvent {
+ /**
+ * The output item that was marked done.
+ */
+ item: ResponseOutputItem;
+
+ /**
+ * The index of the output item that was marked done.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always `response.output_item.done`.
+ */
+ type: 'response.output_item.done';
+}
+
+/**
+ * An output message from the model.
+ */
+export interface ResponseOutputMessage {
+ /**
+ * The unique ID of the output message.
+ */
id: string;
/**
@@ -2117,9 +3185,121 @@ export namespace ResponseOutputText {
}
}
+/**
+ * Emitted when an annotation is added to output text content.
+ */
+export interface ResponseOutputTextAnnotationAddedEvent {
+ /**
+ * The annotation object being added. (See annotation schema for details.)
+ */
+ annotation: unknown;
+
+ /**
+ * The index of the annotation within the content part.
+ */
+ annotation_index: number;
+
+ /**
+ * The index of the content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The unique identifier of the item to which the annotation is being added.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.output_text_annotation.added'.
+ */
+ type: 'response.output_text_annotation.added';
+}
+
+/**
+ * Emitted when a response is queued and waiting to be processed.
+ */
+export interface ResponseQueuedEvent {
+ /**
+ * The full response object that is queued.
+ */
+ response: Response;
+
+ /**
+ * The type of the event. Always 'response.queued'.
+ */
+ type: 'response.queued';
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the reasoning content.
+ */
+export interface ResponseReasoningDeltaEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The partial update to the reasoning content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which reasoning is being updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning.delta'.
+ */
+ type: 'response.reasoning.delta';
+}
+
+/**
+ * Emitted when the reasoning content is finalized for an item.
+ */
+export interface ResponseReasoningDoneEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The unique identifier of the item for which reasoning is finalized.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The finalized reasoning text.
+ */
+ text: string;
+
+ /**
+ * The type of the event. Always 'response.reasoning.done'.
+ */
+ type: 'response.reasoning.done';
+}
+
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +3317,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2158,6 +3344,67 @@ export namespace ResponseReasoningItem {
}
}
+/**
+ * Emitted when there is a delta (partial update) to the reasoning summary content.
+ */
+export interface ResponseReasoningSummaryDeltaEvent {
+ /**
+ * The partial update to the reasoning summary content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which the reasoning summary is being
+ * updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.delta'.
+ */
+ type: 'response.reasoning_summary.delta';
+}
+
+/**
+ * Emitted when the reasoning summary content is finalized for an item.
+ */
+export interface ResponseReasoningSummaryDoneEvent {
+ /**
+ * The unique identifier of the item for which the reasoning summary is finalized.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The finalized reasoning summary text.
+ */
+ text: string;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.done'.
+ */
+ type: 'response.reasoning_summary.done';
+}
+
/**
* Emitted when a new reasoning summary part is added.
*/
@@ -2374,9 +3621,9 @@ export interface ResponseRefusalDoneEvent {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
-export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'incomplete';
+export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'cancelled' | 'queued' | 'incomplete';
,/**
* Emitted when there is a partial audio response.
@@ -2417,7 +3664,25 @@ export type ResponseStreamEvent =
| ResponseTextDoneEvent
| ResponseWebSearchCallCompletedEvent
| ResponseWebSearchCallInProgressEvent
- | ResponseWebSearchCallSearchingEvent;
+ | ResponseWebSearchCallSearchingEvent
+ | ResponseImageGenCallCompletedEvent
+ | ResponseImageGenCallGeneratingEvent
+ | ResponseImageGenCallInProgressEvent
+ | ResponseImageGenCallPartialImageEvent
+ | ResponseMcpCallArgumentsDeltaEvent
+ | ResponseMcpCallArgumentsDoneEvent
+ | ResponseMcpCallCompletedEvent
+ | ResponseMcpCallFailedEvent
+ | ResponseMcpCallInProgressEvent
+ | ResponseMcpListToolsCompletedEvent
+ | ResponseMcpListToolsFailedEvent
+ | ResponseMcpListToolsInProgressEvent
+ | ResponseOutputTextAnnotationAddedEvent
+ | ResponseQueuedEvent
+ | ResponseReasoningDeltaEvent
+ | ResponseReasoningDoneEvent
+ | ResponseReasoningSummaryDeltaEvent
+ | ResponseReasoningSummaryDoneEvent;
/**
* Emitted when a text annotation is added.
@@ -2730,11 +3995,231 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool =
+ | FunctionTool
+ | FileSearchTool
+ | WebSearchTool
+ | ComputerTool
+ | Tool.Mcp
+ | Tool.CodeInterpreter
+ | Tool.ImageGeneration
+ | Tool.LocalShell;
+
+export namespace Tool {
+ /**
+ * Give the model access to additional tools via remote Model Context Protocol
+ * (MCP) servers.
+ * [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp).
+ */
+ export interface Mcp {
+ /**
+ * A label for this MCP server, used to identify it in tool calls.
+ */
+ server_label: string;
+
+ /**
+ * The URL for the MCP server.
+ */
+ server_url: string;
+
+ /**
+ * The type of the MCP tool. Always `mcp`.
+ */
+ type: 'mcp';
+
+ /**
+ * List of allowed tool names or a filter object.
+ */
+ allowed_tools?: Array<string> | Mcp.McpAllowedToolsFilter | null;
+
+ /**
+ * Optional HTTP headers to send to the MCP server. Use for authentication or other
+ * purposes.
+ */
+ headers?: Record<string, string> | null;
+
+ /**
+ * Specify which of the MCP server's tools require approval.
+ */
+ require_approval?: Mcp.McpToolApprovalFilter | 'always' | 'never' | null;
+ }
+
+ export namespace Mcp {
+ /**
+ * A filter object to specify which tools are allowed.
+ */
+ export interface McpAllowedToolsFilter {
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export interface McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ always?: McpToolApprovalFilter.Always;
+
+ /**
+ * A list of tools that never require approval.
+ */
+ never?: McpToolApprovalFilter.Never;
+
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export namespace McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ export interface Always {
+ /**
+ * List of tools that require approval.
+ */
+ tool_names?: Array<string>;
+ }
+
+ /**
+ * A list of tools that never require approval.
+ */
+ export interface Never {
+ /**
+ * List of tools that do not require approval.
+ */
+ tool_names?: Array<string>;
+ }
+ }
+ }
+
+ /**
+ * A tool that runs Python code to help generate a response to a prompt.
+ */
+ export interface CodeInterpreter {
+ /**
+ * The code interpreter container. Can be a container ID or an object that
+ * specifies uploaded file IDs to make available to your code.
+ */
+ container: string | CodeInterpreter.CodeInterpreterToolAuto;
+
+ /**
+ * The type of the code interpreter tool. Always `code_interpreter`.
+ */
+ type: 'code_interpreter';
+ }
+
+ export namespace CodeInterpreter {
+ /**
+ * Configuration for a code interpreter container. Optionally specify the IDs of
+ * the files to run the code on.
+ */
+ export interface CodeInterpreterToolAuto {
+ /**
+ * Always `auto`.
+ */
+ type: 'auto';
+
+ /**
+ * An optional list of uploaded files to make available to your code.
+ */
+ file_ids?: Array<string>;
+ }
+ }
+
+ /**
+ * A tool that generates images using a model like `gpt-image-1`.
+ */
+ export interface ImageGeneration {
+ /**
+ * The type of the image generation tool. Always `image_generation`.
+ */
+ type: 'image_generation';
+
+ /**
+ * Background type for the generated image. One of `transparent`, `opaque`, or
+ * `auto`. Default: `auto`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto';
+
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ input_image_mask?: ImageGeneration.InputImageMask;
+
+ /**
+ * The image generation model to use. Default: `gpt-image-1`.
+ */
+ model?: 'gpt-image-1';
+
+ /**
+ * Moderation level for the generated image. Default: `auto`.
+ */
+ moderation?: 'auto' | 'low';
+
+ /**
+ * Compression level for the output image. Default: 100.
+ */
+ output_compression?: number;
+
+ /**
+ * The output format of the generated image. One of `png`, `webp`, or `jpeg`.
+ * Default: `png`.
+ */
+ output_format?: 'png' | 'webp' | 'jpeg';
+
+ /**
+ * Number of partial images to generate in streaming mode, from 0 (default value)
+ * to 3.
+ */
+ partial_images?: number;
+
+ /**
+ * The quality of the generated image. One of `low`, `medium`, `high`, or `auto`.
+ * Default: `auto`.
+ */
+ quality?: 'low' | 'medium' | 'high' | 'auto';
+
+ /**
+ * The size of the generated image. One of `1024x1024`, `1024x1536`, `1536x1024`,
+ * or `auto`. Default: `auto`.
+ */
+ size?: '1024x1024' | '1024x1536' | '1536x1024' | 'auto';
+ }
+
+ export namespace ImageGeneration {
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ export interface InputImageMask {
+ /**
+ * File ID for the mask image.
+ */
+ file_id?: string;
+
+ /**
+ * Base64-encoded mask image.
+ */
+ image_url?: string;
+ }
+ }
+
+ /**
+ * A tool that allows the model to execute shell commands in a local environment.
+ */
+ export interface LocalShell {
+ /**
+ * The type of the local shell tool. Always `local_shell`.
+ */
+ type: 'local_shell';
+ }
+}
/**
* Use this option to force the model to call a specific function.
@@ -2777,8 +4262,18 @@ export interface ToolChoiceTypes {
* - `file_search`
* - `web_search_preview`
* - `computer_use_preview`
- */
- type: 'file_search' | 'web_search_preview' | 'computer_use_preview' | 'web_search_preview_2025_03_11';
+ * - `code_interpreter`
+ * - `mcp`
+ * - `image_generation`
+ */
+ type:
+ | 'file_search'
+ | 'web_search_preview'
+ | 'computer_use_preview'
+ | 'web_search_preview_2025_03_11'
+ | 'image_generation'
+ | 'code_interpreter'
+ | 'mcp';
}
/**
@@ -2788,10 +4283,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +4294,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +4313,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2860,6 +4359,12 @@ export interface ResponseCreateParamsBase {
*/
model: Shared.ResponsesModel;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* Specify additional output data to include in the model response. Currently
* supported values are:
@@ -2869,6 +4374,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
@@ -3104,6 +4614,10 @@ export declare namespace Responses {
type ResponseFunctionToolCallItem as ResponseFunctionToolCallItem,
type ResponseFunctionToolCallOutputItem as ResponseFunctionToolCallOutputItem,
type ResponseFunctionWebSearch as ResponseFunctionWebSearch,
+ type ResponseImageGenCallCompletedEvent as ResponseImageGenCallCompletedEvent,
+ type ResponseImageGenCallGeneratingEvent as ResponseImageGenCallGeneratingEvent,
+ type ResponseImageGenCallInProgressEvent as ResponseImageGenCallInProgressEvent,
+ type ResponseImageGenCallPartialImageEvent as ResponseImageGenCallPartialImageEvent,
type ResponseInProgressEvent as ResponseInProgressEvent,
type ResponseIncludable as ResponseIncludable,
type ResponseIncompleteEvent as ResponseIncompleteEvent,
@@ -3117,6 +4631,14 @@ export declare namespace Responses {
type ResponseInputMessageItem as ResponseInputMessageItem,
type ResponseInputText as ResponseInputText,
type ResponseItem as ResponseItem,
+ type ResponseMcpCallArgumentsDeltaEvent as ResponseMcpCallArgumentsDeltaEvent,
+ type ResponseMcpCallArgumentsDoneEvent as ResponseMcpCallArgumentsDoneEvent,
+ type ResponseMcpCallCompletedEvent as ResponseMcpCallCompletedEvent,
+ type ResponseMcpCallFailedEvent as ResponseMcpCallFailedEvent,
+ type ResponseMcpCallInProgressEvent as ResponseMcpCallInProgressEvent,
+ type ResponseMcpListToolsCompletedEvent as ResponseMcpListToolsCompletedEvent,
+ type ResponseMcpListToolsFailedEvent as ResponseMcpListToolsFailedEvent,
+ type ResponseMcpListToolsInProgressEvent as ResponseMcpListToolsInProgressEvent,
type ResponseOutputAudio as ResponseOutputAudio,
type ResponseOutputItem as ResponseOutputItem,
type ResponseOutputItemAddedEvent as ResponseOutputItemAddedEvent,
@@ -3124,7 +4646,13 @@ export declare namespace Responses {
type ResponseOutputMessage as ResponseOutputMessage,
type ResponseOutputRefusal as ResponseOutputRefusal,
type ResponseOutputText as ResponseOutputText,
+ type ResponseOutputTextAnnotationAddedEvent as ResponseOutputTextAnnotationAddedEvent,
+ type ResponseQueuedEvent as ResponseQueuedEvent,
+ type ResponseReasoningDeltaEvent as ResponseReasoningDeltaEvent,
+ type ResponseReasoningDoneEvent as ResponseReasoningDoneEvent,
type ResponseReasoningItem as ResponseReasoningItem,
+ type ResponseReasoningSummaryDeltaEvent as ResponseReasoningSummaryDeltaEvent,
+ type ResponseReasoningSummaryDoneEvent as ResponseReasoningSummaryDoneEvent,
type ResponseReasoningSummaryPartAddedEvent as ResponseReasoningSummaryPartAddedEvent,
type ResponseReasoningSummaryPartDoneEvent as ResponseReasoningSummaryPartDoneEvent,
type ResponseReasoningSummaryTextDeltaEvent as ResponseReasoningSummaryTextDeltaEvent,
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..ff06c96c3 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.101.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responseProm,ise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
diff --git tests/api-resources/responses/responses.test.ts tests/api-resources/responses/responses.test.ts
index cf7e9cf3c..1b2cf7fc4 100644
--- tests/api-resources/responses/responses.test.ts
+++ tests/api-resources/responses/responses.test.ts
@@ -24,6 +24,7 @@ describe('resource responses', () => {
const response = await client.responses.create({
input: 'string',
model: 'gpt-4o',
+ background: true,
include: ['file_search_call.results'],
instructions: 'instructions',
max_output_tokens: 0,
@@ -39,11 +40,11 @@ describe('resource responses', () => {
tool_choice: 'none',
tools: [
{
- type: 'file_search',
- vector_store_ids: ['string'],
- filters: { key: 'key', type: 'eq', value: 'string' },
- max_num_results: 0,
- ranking_options: { ranker: 'auto', score_threshold: 0 },
+ name: 'name',
+ parameters: { foo: 'bar' },
+ strict: true,
+ type: 'function',
+ description: 'description',
},
],
top_p: 1,
Description
This PR updates the openai-node SDK to version 4.101.0 from 4.96.2. It includes API updates for new features, Node 20 upgrades in GitHub workflows, bug fixes, and documentation improvements.
Changes
Changes
CI Configuration
- Updated Node.js version from 18 to 20 in several GitHub Actions workflows:
-
.github/workflows/ci.yml -
.github/workflows/publish-jsr.yml -
.github/workflows/publish-npm.yml
-
Version Updates
- Bumped version numbers in several files:
-
.release-please-manifest.json -
jsr.json -
package.json -
src/version.ts
-
API Additions and Updates
- Added reinforcement fine-tuning API support
- Added new image generation capabilities (including background transparency options)
- Added Graders API for evaluations
- Added MCP (Model Context Protocol) tools
- Added new response event types for reasoning and image generation
- Added pause/resume functionality to fine-tuning jobs
- Updated embedding model documentation
- Added documentation examples to many API methods
Documentation
- Fixed typos and improved grammar in documentation:
- Fixed "procesing" to "processing" in examples
- Improved grammar in SECURITY.md
- Added JSDoc examples to many API methods
Tests
- Added new tests for the new API functionality:
- Added tests for fine-tuning alpha graders
- Updated job tests for pause/resume functionality
- Updated image tests for new parameters
sequenceDiagram
Developer->>GitHub: Submit PR to upgrade Node.js to v20
Developer->>GitHub: Add reinforcement fine-tuning API support
Developer->>GitHub: Add Graders API for evaluations
Developer->>GitHub: Add MCP tools functionality
Developer->>GitHub: Add new image generation capabilities
Developer->>GitHub: Update documentation with examples
Developer->>GitHub: Add pause/resume functionality to fine-tuning jobs
Developer->>GitHub: Fix typos and improve grammar
GitHub->>CI: Run tests with Node.js v20
CI->>GitHub: Tests pass
GitHub->>openai-node: Merge changes to main
openai-node->>npm: Publish v4.101.0
openai-node->>JSR: Publish v4.101.0
openai debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..514215a1e 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.101.0"
}
diff --git .stats.yml .stats.yml
index d92408173..4b4f19c91 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-a5651cb97f86d1e2531af6aef8c5230f1ea350560fbae790ca2e481b30a6c217.yml
+openapi_spec_hash: 66a5104fd3bb43383cf919225df7a6fd
+config_hash: bb657c3fed232a56930035de3aaed936
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..1e427d912 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,80 @@
# Changelog
+## 4.101.0 (2025-05-21)
+
+Full Changelog: [v4.100.0...v4.101.0](https://github.com/openai/openai-node/compare/v4.100.0...v4.101.0)
+
+### Features
+
+* **api:** new API tools ([fb4014f](https://github.com/openai/openai-node/commit/fb4014ffac7b220d37bd03c94fa745386b010bf0))
+
+
+### Chores
+
+* **docs:** grammar improvements ([7761cfb](https://github.com/openai/openai-node/commit/7761cfb0a8a56d056a73c046a6a613f66ada4abe))
+* **internal:** version bump ([b40e830](https://github.com/openai/openai-node/commit/b40e8302ec11683b6a360a050236ad1284afc760))
+
+## 4.100.0 (2025-05-16)
+
+Full Changelog: [v4.99.0...v4.100.0](https://github.com/openai/openai-node/compare/v4.99.0...v4.100.0)
+
+### Features
+
+* **api:** further updates for evals API ([3f6f248](https://github.com/openai/openai-node/commit/3f6f248191b45015924be76fd5154d149c4ed8a0))
+
+
+### Chores
+
+* **internal:** version bump ([5123fe0](https://github.com/openai/openai-node/commit/5123fe08a56f3d0040b1cc67129382f3eacc3cca))
+
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git SECURITY.md SECURITY.md
index 3b3bd8a66..4adb0c54f 100644
--- SECURITY.md
+++ SECURITY.md
@@ -16,13 +16,13 @@ before making any information public.
## Reporting Non-SDK Related Security Issues
If you encounter security issues that are not directly related to SDKs but pertain to the services
-or products provided by OpenAI please follow the respective company's security reporting guidelines.
+or products provided by OpenAI, please follow the respective company's security reporting guidelines.
### OpenAI Terms and Policies
Our Security Policy can be found at [Security Policy URL](https://openai.com/policies/coordinated-vulnerability-disclosure-policy).
-Please contact [email protected] for any questions or concerns regarding security of our services.
+Please contact [email protected] for any questions or concerns regarding the security of our services.
---
diff --git api.md api.md
index 49e6548a8..f7fb47e3e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -605,6 +645,10 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionWebSearch</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallGeneratingEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallPartialImageEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncludable</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncompleteEvent</a></code>
@@ -618,6 +662,14 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseInputMessageItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInputText</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDoneEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputAudio</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItemAddedEvent</a></code>
@@ -625,7 +677,13 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputMessage</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputRefusal</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputText</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseOutputTextAnnotationAddedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseQueuedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartAddedEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryTextDeltaEvent</a></code>
@@ -669,10 +727,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..42bf0a235 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.101.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..e8cd7277c 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.101.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/realtime.ts src/resources/beta/realtime/realtime.ts
index 5012b1edd..26fba883e 100644
--- src/resources/beta/realtime/realtime.ts
+++ src/resources/beta/realtime/realtime.ts
@@ -2571,7 +2571,7 @@ export interface TranscriptionSessionUpdatedEvent {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
session: TranscriptionSessionsAPI.TranscriptionSession;
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..83e8c47ad 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
@@ -29,7 +35,7 @@ export class TranscriptionSessions extends APIResource {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
export interface TranscriptionSession {
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..05a656619 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -27,7 +28,8 @@ export class Evals extends APIResource {
/**
* Create the structure of an evaluation that can be used to test a model's
- * performance. An evaluation is a set of testing criteria and a datasource. After
+ * performance. An evaluation is a set of testing criteria and the config for a
+ * data source, which dictates the schema of the data used in the evaluation. After
* creating an evaluation, you can run it on different models and model parameters.
* We support several types of graders and datasources. For more information, see
* the [Evals guide](https://platform.openai.com/docs/guides/evals).
@@ -104,88 +106,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -210,90 +131,13 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalCreateResponse {
/**
@@ -309,7 +153,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +182,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +244,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -465,7 +258,7 @@ export namespace EvalCreateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalRetrieveResponse {
/**
@@ -481,7 +274,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +303,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +365,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -637,7 +379,7 @@ export namespace EvalRetrieveResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalUpdateResponse {
/**
@@ -653,7 +395,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +424,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +486,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
*, The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -809,7 +500,7 @@ export namespace EvalUpdateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalListResponse {
/**
@@ -825,7 +516,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +545,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +607,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -985,17 +625,21 @@ export interface EvalDeleteResponse {
export interface EvalCreateParams {
/**
- * The configuration for the data source used for the evaluation runs.
+ * The configuration for the data source used for the evaluation runs. Dictates the
+ * schema of the data used in the evaluation.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
- * A list of graders for all eval runs in this group.
+ * A list of graders for all eval runs in this group. Graders can reference
+ * variables in the data source using double curly braces notation, like
+ * `{{item.variable_name}}`. To reference the model's output, use the `sample`
+ * namespace (ie, `{{sample.output_text}}`).
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +688,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +703,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored_completions`.
+ */
+ type: 'stored_completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1067,7 +725,7 @@ export namespace EvalCreateParams {
export interface LabelModel {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
input: Array<LabelModel.SimpleInputMessage | LabelModel.EvalItem>;
@@ -1154,29 +812,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +834,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +880,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..e761e2160 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -19,7 +19,9 @@ export class Runs extends APIResource {
outputItems: OutputItemsAPI.OutputItems = new OutputItemsAPI.OutputItems(this._client);
/**
- * Create a new evaluation run. This is the endpoint that will kick off grading.
+ * Kicks off a new run for a given evaluation, specifying the data source, and what
+ * model configuration to use to test. The datasource will be validated against the
+ * schema specified in the config of the evaluation.
*/
create(
evalId: string,
@@ -85,7 +87,7 @@ export class RunListResponsesPage extends CursorPage<RunListResponse> {}
*/
export interface CreateEvalCompletionsRunDataSource {
/**
- * A StoredCompletionsRunDataSource configuration describing a set of filters
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalCompletionsRunDataSource.FileContent
@@ -97,6 +99,12 @@ export interface CreateEvalCompletionsRunDataSource {
*/
type: 'completions';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalCompletionsRunDataSource.Template
| CreateEvalCompletionsRunDataSource.ItemReference;
@@ -185,7 +193,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<ResponsesAPI.EasyInputMessage | Template.Message>;
@@ -241,7 +249,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"
*/
item_reference: string;
@@ -279,6 +287,9 @@ export namespace CreateEvalCompletionsRunDataSource {
* eval
*/
export interface CreateEvalJSONLRunDataSource {
+ /**
+ * Determines what populates the `item` namespace in the data source.
+ */
source: CreateEvalJSONLRunDataSource.FileContent | CreateEvalJSONLRunDataSource.FileID;
/**
@@ -356,7 +367,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +434,34 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +503,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -505,14 +516,8 @@ export namespace RunCreateResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +544,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -553,7 +563,7 @@ export namespace RunCreateResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -621,7 +631,7 @@ export namespace RunCreateResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -749,7 +759,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +826,34 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +895,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -898,14 +908,8 @@ export namespace RunRetrieveResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +936,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -946,7 +955,7 @@ export namespace RunRetrieveResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1014,7 +1023,7 @@ export namespace RunRetrieveResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1139,10 +1148,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1215,34 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1284,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1291,14 +1297,8 @@ export namespace RunListResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1325,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1339,7 +1344,7 @@ export namespace RunListResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1407,7 +1412,7 @@ export namespace RunListResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1543,7 +1548,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1615,34 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1684,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1692,14 +1697,8 @@ export namespace RunCancelResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1725,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1740,7 +1744,7 @@ export namespace RunCancelResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1808,7 +1812,7 @@ export namespace RunCancelResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1947,7 +1951,7 @@ export namespace RunCreateParams {
*/
export interface CreateEvalResponsesRunDataSource {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalResponsesRunDataSource.FileContent
@@ -1955,10 +1959,16 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalResponsesRunDataSource.Template
| CreateEvalResponsesRunDataSource.ItemReference;
@@ -2013,12 +2023,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2032,14 +2036,8 @@ export namespace RunCreateParams {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2064,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -2080,7 +2083,7 @@ export namespace RunCreateParams {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -2148,7 +2151,7 @@ export namespace RunCreateParams {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | Tex,tSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..fdf51f224 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -37,7 +37,13 @@ export type ParsedResponseOutputItem<ParsedT> =
| ResponseFileSearchToolCall
| ResponseFunctionWebSearch
| ResponseComputerToolCall
- | ResponseReasoningItem;
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
export interface ParsedResponse<ParsedT> extends Response {
output: Array<ParsedResponseOutputItem<ParsedT>>;
@@ -61,6 +67,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +104,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +131,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +186,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +237,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +286,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -392,6 +420,12 @@ export interface Response {
*/
top_p: number | null;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* An upper bound for the number of tokens that can be generated for a response,
* including visible output tokens and
@@ -437,7 +471,7 @@ export interface Response {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
status?: ResponseStatus;
@@ -665,6 +699,11 @@ export interface ResponseCodeInterpreterToolCall {
* The type of the code interpreter tool call. Always `code_interpreter_call`.
*/
type: 'code_interpreter_call';
+
+ /**
+ * The ID of the container used to run the code.
+ */
+ container_id?: string;
}
export namespace ResponseCodeInterpreterToolCall {
@@ -1557,6 +1596,114 @@ export interface ResponseFunctionWebSearch {
type: 'web_search_call';
}
+/**
+ * Emitted when an image generation tool call has completed and the final image is
+ * available.
+ */
+export interface ResponseImageGenCallCompletedEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.completed'.
+ */
+ type: 'response.image_generation_call.completed';
+}
+
+/**
+ * Emitted when an image generation tool call is actively generating an image
+ * (intermediate state).
+ */
+export interface ResponseImageGenCallGeneratingEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.generating'.
+ */
+ type: 'response.image_generation_call.generating';
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number?: number;
+}
+
+/**
+ * Emitted when an image generation tool call is in progress.
+ */
+export interface ResponseImageGenCallInProgressEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.in_progress'.
+ */
+ type: 'response.image_generation_call.in_progress';
+}
+
+/**
+ * Emitted when a partial image is available during image generation streaming.
+ */
+export interface ResponseImageGenCallPartialImageEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * Base64-encoded partial image data, suitable for rendering as an image.
+ */
+ partial_image_b64: string;
+
+ /**
+ * 0-based index for the partial image (backend is 1-based, but this is 0-based for
+ * the user).
+ */
+ partial_image_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.partial_image'.
+ */
+ type: 'response.image_generation_call.partial_image';
+}
+
/**
* Emitted when the response is in progress.
*/
@@ -1581,11 +1728,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1803,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1820,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1704,6 +1857,14 @@ export type ResponseInputItem =
| ResponseFunctionToolCall
| ResponseInputItem.FunctionCallOutput
| ResponseReasoningItem
+ | ResponseInputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseInputItem.LocalShellCall
+ | ResponseInputItem.LocalShellCallOutput
+ | ResponseInputItem.McpListTools
+ | ResponseInputItem.McpApprovalRequest
+ | ResponseInputItem.McpApprovalResponse
+ | ResponseInputItem.McpCall
| ResponseInputItem.ItemReference;
export namespace ResponseInputItem {
@@ -1758,19 +1919,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1947,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,172 +1979,1079 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
- * An internal identifier for an item to reference.
+ * An image generation request made by the model.
*/
- export interface ItemReference {
+ export interface ImageGenerationCall {
/**
- * The ID of the item to reference.
+ * The unique ID of the image generation call.
*/
id: string;
/**
- * The type of item to reference. Always `item_reference`.
+ * The generated image encoded in base64.
*/
- type: 'item_reference';
- }
-}
+ result: string | null;
-/**
- * A list of one or many input items to the model, containing different content
- * types.
- */
-export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
-export interface ResponseInputMessageItem {
- /**
- * The unique ID of the message input.
- */
- id: string;
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
/**
- * A list of one or many input items to the model, containing different content
- * types.
+ * A tool call to run a command on the local shell.
*/
- content: ResponseInputMessageContentList;
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
- /**
- * The role of the message input. One of `user`, `system`, or `developer`.
- */
- role: 'user' | 'system' | 'developer';
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
- /**
- * The status of item. One of `in_progress`, `completed`, or `incomplete`.
- * Populated when items are returned via API.
- */
- status?: 'in_progress' | 'completed' | 'incomplete';
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
- /**
- * The type of the message input. Always set to `message`.
- */
- type?: 'message';
-}
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
-/**
- * A text input to the model.
- */
-export interface ResponseInputText {
- /**
- * The text input to the model.
- */
- text: string;
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
- /**
- * The type of the input item. Always `input_text`.
- */
- type: 'input_text';
-}
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
-/**
- * Content item used to generate a response.
- */
-export type ResponseItem =
- | ResponseInputMessageItem
- | ResponseOutputMessage
- | ResponseFileSearchToolCall
- | ResponseComputerToolCall
- | ResponseComputerToolCallOutputItem
- | ResponseFunctionWebSearch
- | ResponseFunctionToolCallItem
- | ResponseFunctionToolCallOutputItem;
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
-/**
- * An audio output from the model.
- */
-export interface ResponseOutputAudio {
- /**
- * Base64-encoded audio data from the model.
- */
- data: string;
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
- /**
- * The transcript of the audio data from the model.
- */
- transcript: string;
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
- /**
- * The type of the output audio. Always `output_audio`.
- */
- type: 'output_audio';
-}
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
-/**
- * An output message from the model.
- */
-export type ResponseOutputItem =
- | ResponseOutputMessage
- | ResponseFileSearchToolCall
- | ResponseFunctionToolCall
- | ResponseFunctionWebSearch
- | ResponseComputerToolCall
- | ResponseReasoningItem;
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
-/**
- * Emitted when a new output item is added.
- */
-export interface ResponseOutputItemAddedEvent {
/**
- * The output item that was added.
+ * The output of a local shell tool call.
*/
- item: ResponseOutputItem;
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
- /**
- * The index of the output item that was added.
- */
- output_index: number;
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
- /**
- * The type of the event. Always `response.output_item.added`.
- */
- type: 'response.output_item.added';
-}
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
-/**
- * Emitted when an output item is marked done.
- */
-export interface ResponseOutputItemDoneEvent {
- /**
- * The output item that was marked done.
- */
- item: ResponseOutputItem;
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
/**
- * The index of the output item that was marked done.
+ * A list of tools available on an MCP server.
*/
- output_index: number;
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
- /**
- * The type of the event. Always `response.output_item.done`.
- */
- type: 'response.output_item.done';
-}
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
-/**
- * An output message from the model.
- */
-export interface ResponseOutputMessage {
- /**
- * The unique ID of the output message.
- */
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * The unique ID of the approval response
+ */
+ id?: string | null;
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * An internal identifier for an item to reference.
+ */
+ export interface ItemReference {
+ /**
+ * The ID of the item to reference.
+ */
+ id: string;
+
+ /**
+ * The type of item to reference. Always `item_reference`.
+ */
+ type?: 'item_reference' | null;
+ }
+}
+
+/**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+
+export interface ResponseInputMessageItem {
+ /**
+ * The unique ID of the message input.
+ */
+ id: string;
+
+ /**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+ content: ResponseInputMessageContentList;
+
+ /**
+ * The role of the message input. One of `user`, `system`, or `developer`.
+ */
+ role: 'user' | 'system' | 'developer';
+
+ /**
+ * The status of item. One of `in_progress`, `completed`, or `incomplete`.
+ * Populated when items are returned via API.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the message input. Always set to `message`.
+ */
+ type?: 'message';
+}
+
+/**
+ * A text input to the model.
+ */
+export interface ResponseInputText {
+ /**
+ * The text input to the model.
+ */
+ text: string;
+
+ /**
+ * The type of the input item. Always `input_text`.
+ */
+ type: 'input_text';
+}
+
+/**
+ * Content item used to generate a response.
+ */
+export type ResponseItem =
+ | ResponseInputMessageItem
+ | ResponseOutputMessage
+ | ResponseFileSearchToolCall
+ | ResponseComputerToolCall
+ | ResponseComputerToolCallOutputItem
+ | ResponseFunctionWebSearch
+ | ResponseFunctionToolCallItem
+ | ResponseFunctionToolCallOutputItem
+ | ResponseItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseItem.LocalShellCall
+ | ResponseItem.LocalShellCallOutput
+ | ResponseItem.McpListTools
+ | ResponseItem.McpApprovalRequest
+ | ResponseItem.McpApprovalResponse
+ | ResponseItem.McpCall;
+
+export namespace ResponseItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * The output of a local shell tool call.
+ */
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
+
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
+
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The unique ID of the approval response
+ */
+ id: string;
+
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the arguments of an MCP tool
+ * call.
+ */
+export interface ResponseMcpCallArgumentsDeltaEvent {
+ /**
+ * The partial update to the arguments for the MCP tool call.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_delta'.
+ */
+ type: 'response.mcp_call.arguments_delta';
+}
+
+/**
+ * Emitted when the arguments for an MCP tool call are finalized.
+ */
+export interface ResponseMcpCallArgumentsDoneEvent {
+ /**
+ * The finalized arguments for the MCP tool call.
+ */
+ arguments: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_done'.
+ */
+ type: 'response.mcp_call.arguments_done';
+}
+
+/**
+ * Emitted when an MCP tool call has completed successfully.
+ */
+export interface ResponseMcpCallCompletedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_call.completed'.
+ */
+ type: 'response.mcp_call.completed';
+}
+
+/**
+ * Emitted when an MCP tool call has failed.
+ */
+export interface ResponseMcpCallFailedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_call.failed'.
+ */
+ type: 'response.mcp_call.failed';
+}
+
+/**
+ * Emitted when an MCP tool call is in progress.
+ */
+export interface ResponseMcpCallInProgressEvent {
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.in_progress'.
+ */
+ type: 'response.mcp_call.in_progress';
+}
+
+/**
+ * Emitted when the list of available MCP tools has been successfully retrieved.
+ */
+export interface ResponseMcpListToolsCompletedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.completed'.
+ */
+ type: 'response.mcp_list_tools.completed';
+}
+
+/**
+ * Emitted when the attempt to list available MCP tools has failed.
+ */
+export interface ResponseMcpListToolsFailedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.failed'.
+ */
+ type: 'response.mcp_list_tools.failed';
+}
+
+/**
+ * Emitted when the system is in the process of retrieving the list of available
+ * MCP tools.
+ */
+export interface ResponseMcpListToolsInProgressEvent {
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.in_progress'.
+ */
+ type: 'response.mcp_list_tools.in_progress';
+}
+
+/**
+ * An audio output from the model.
+ */
+export interface ResponseOutputAudio {
+ /**
+ * Base64-encoded audio data from the model.
+ */
+ data: string;
+
+ /**
+ * The transcript of the audio data from the model.
+ */
+ transcript: string;
+
+ /**
+ * The type of the output audio. Always `output_audio`.
+ */
+ type: 'output_audio';
+}
+
+/**
+ * An output message from the model.
+ */
+export type ResponseOutputItem =
+ | ResponseOutputMessage
+ | ResponseFileSearchToolCall
+ | ResponseFunctionToolCall
+ | ResponseFunctionWebSearch
+ | ResponseComputerToolCall
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
+
+export namespace ResponseOutputItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+}
+
+/**
+ * Emitted when a new output item is added.
+ */
+export interface ResponseOutputItemAddedEvent {
+ /**
+ * The output item that was added.
+ */
+ item: ResponseOutputItem;
+
+ /**
+ * The index of the output item that was added.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always `response.output_item.added`.
+ */
+ type: 'response.output_item.added';
+}
+
+/**
+ * Emitted when an output item is marked done.
+ */
+export interface ResponseOutputItemDoneEvent {
+ /**
+ * The output item that was marked done.
+ */
+ item: ResponseOutputItem;
+
+ /**
+ * The index of the output item that was marked done.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always `response.output_item.done`.
+ */
+ type: 'response.output_item.done';
+}
+
+/**
+ * An output message from the model.
+ */
+export interface ResponseOutputMessage {
+ /**
+ * The unique ID of the output message.
+ */
id: string;
/**
@@ -2117,9 +3185,121 @@ export namespace ResponseOutputText {
}
}
+/**
+ * Emitted when an annotation is added to output text content.
+ */
+export interface ResponseOutputTextAnnotationAddedEvent {
+ /**
+ * The annotation object being added. (See annotation schema for details.)
+ */
+ annotation: unknown;
+
+ /**
+ * The index of the annotation within the content part.
+ */
+ annotation_index: number;
+
+ /**
+ * The index of the content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The unique identifier of the item to which the annotation is being added.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.output_text_annotation.added'.
+ */
+ type: 'response.output_text_annotation.added';
+}
+
+/**
+ * Emitted when a response is queued and waiting to be processed.
+ */
+export interface ResponseQueuedEvent {
+ /**
+ * The full response object that is queued.
+ */
+ response: Response;
+
+ /**
+ * The type of the event. Always 'response.queued'.
+ */
+ type: 'response.queued';
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the reasoning content.
+ */
+export interface ResponseReasoningDeltaEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The partial update to the reasoning content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which reasoning is being updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning.delta'.
+ */
+ type: 'response.reasoning.delta';
+}
+
+/**
+ * Emitted when the reasoning content is finalized for an item.
+ */
+export interface ResponseReasoningDoneEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The unique identifier of the item for which reasoning is finalized.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The finalized reasoning text.
+ */
+ text: string;
+
+ /**
+ * The type of the event. Always 'response.reasoning.done'.
+ */
+ type: 'response.reasoning.done';
+}
+
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +3317,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2158,6 +3344,67 @@ export namespace ResponseReasoningItem {
}
}
+/**
+ * Emitted when there is a delta (partial update) to the reasoning summary content.
+ */
+export interface ResponseReasoningSummaryDeltaEvent {
+ /**
+ * The partial update to the reasoning summary content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which the reasoning summary is being
+ * updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.delta'.
+ */
+ type: 'response.reasoning_summary.delta';
+}
+
+/**
+ * Emitted when the reasoning summary content is finalized for an item.
+ */
+export interface ResponseReasoningSummaryDoneEvent {
+ /**
+ * The unique identifier of the item for which the reasoning summary is finalized.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The finalized reasoning summary text.
+ */
+ text: string;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.done'.
+ */
+ type: 'response.reasoning_summary.done';
+}
+
/**
* Emitted when a new reasoning summary part is added.
*/
@@ -2374,9 +3621,9 @@ export interface ResponseRefusalDoneEvent {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
-export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'incomplete';
+export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'cancelled' | 'queued' | 'incomplete';
/**
* Emitted when there is a partial audio response.
@@ -2417,7 +3664,25 @@ export type ResponseStreamEvent =
| ResponseTextDoneEvent
| ResponseWebSearchCallCompletedEvent
| ResponseWebSearchCallInProgressEvent
- | ResponseWebSearchCallSearchingEvent;
+ | ResponseWebSearchCallSearchingEvent
+ | ResponseImageGenCallCompletedEvent
+ | ResponseImageGenCallGeneratingEvent
+ | ResponseImageGenCallInProgressEvent
+ | ResponseImageGenCallPartialImageEvent
+ | ResponseMcpCallArgumentsDeltaEvent
+ | ResponseMcpCallArgumentsDoneEvent
+ | ResponseMcpCallCompletedEvent
+ | ResponseMcpCallFailedEvent
+ | ResponseMcpCallInProgressEvent
+ | ResponseMcpListToolsCompletedEvent
+ | ResponseMcpListToolsFailedEvent
+ | ResponseMcpListToolsInProgressEvent
+ | ResponseOutputTextAnnotationAddedEvent
+ | ResponseQueuedEvent
+ | ResponseReasoningDeltaEvent
+ | ResponseReasoningDoneEvent
+ | ResponseReasoningSummaryDeltaEvent
+ | ResponseReasoningSummaryDoneEvent;
/**
* Emitted when a text annotation is added.
@@ -2730,11 +3995,231 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool =
+ | FunctionTool
+ | FileSearchTool
+ | WebSearchTool
+ | ComputerTool
+ | Tool.Mcp
+ | Tool.CodeInterpreter
+ | Tool.ImageGeneration
+ | Tool.LocalShell;
+
+export namespace Tool {
+ /**
+ * Give the model access to additional tools via remote Model Context Protocol
+ * (MCP) servers.
+ * [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp).
+ */
+ export interface Mcp {
+ /**
+ * A label for this MCP server, used to identify it in tool calls.
+ */
+ server_label: string;
+
+ /**
+ * The URL for the MCP server.
+ */
+ server_url: string;
+
+ /**
+ * The type of the MCP tool. Always `mcp`.
+ */
+ type: 'mcp';
+
+ /**
+ * List of allowed tool names or a filter object.
+ */
+ allowed_tools?: Array<string> | Mcp.McpAllowedToolsFilter | null;
+
+ /**
+ * Optional HTTP headers to send to the MCP server. Use for authentication or other
+ * purposes.
+ */
+ headers?: Record<string, string> | null;
+
+ /**
+ * Specify which of the MCP server's tools require approval.
+ */
+ require_approval?: Mcp.McpToolApprovalFilter | 'always' | 'never' | null;
+ }
+
+ export namespace Mcp {
+ /**
+ * A filter object to specify which tools are allowed.
+ */
+ export interface McpAllowedToolsFilter {
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export interface McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ always?: McpToolApprovalFilter.Always;
+
+ /**
+ * A list of tools that never require approval.
+ */
+ never?: McpToolApprovalFilter.Never;
+
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export namespace McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ export interface Always {
+ /**
+ * List of tools that require approval.
+ */
+ tool_names?: Array<string>;
+ }
+
+ /**
+ * A list of tools that never require approval.
+ */
+ export interface Never {
+ /**
+ * List of tools that do not require approval.
+ */
+ tool_names?: Array<string>;
+ }
+ }
+ }
+
+ /**
+ * A tool that runs Python code to help generate a response to a prompt.
+ */
+ export interface CodeInterpreter {
+ /**
+ * The code interpreter container. Can be a container ID or an object that
+ * specifies uploaded file IDs to, make available to your code.
+ */
+ container: string | CodeInterpreter.CodeInterpreterToolAuto;
+
+ /**
+ * The type of the code interpreter tool. Always `code_interpreter`.
+ */
+ type: 'code_interpreter';
+ }
+
+ export namespace CodeInterpreter {
+ /**
+ * Configuration for a code interpreter container. Optionally specify the IDs of
+ * the files to run the code on.
+ */
+ export interface CodeInterpreterToolAuto {
+ /**
+ * Always `auto`.
+ */
+ type: 'auto';
+
+ /**
+ * An optional list of uploaded files to make available to your code.
+ */
+ file_ids?: Array<string>;
+ }
+ }
+
+ /**
+ * A tool that generates images using a model like `gpt-image-1`.
+ */
+ export interface ImageGeneration {
+ /**
+ * The type of the image generation tool. Always `image_generation`.
+ */
+ type: 'image_generation';
+
+ /**
+ * Background type for the generated image. One of `transparent`, `opaque`, or
+ * `auto`. Default: `auto`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto';
+
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ input_image_mask?: ImageGeneration.InputImageMask;
+
+ /**
+ * The image generation model to use. Default: `gpt-image-1`.
+ */
+ model?: 'gpt-image-1';
+
+ /**
+ * Moderation level for the generated image. Default: `auto`.
+ */
+ moderation?: 'auto' | 'low';
+
+ /**
+ * Compression level for the output image. Default: 100.
+ */
+ output_compression?: number;
+
+ /**
+ * The output format of the generated image. One of `png`, `webp`, or `jpeg`.
+ * Default: `png`.
+ */
+ output_format?: 'png' | 'webp' | 'jpeg';
+
+ /**
+ * Number of partial images to generate in streaming mode, from 0 (default value)
+ * to 3.
+ */
+ partial_images?: number;
+
+ /**
+ * The quality of the generated image. One of `low`, `medium`, `high`, or `auto`.
+ * Default: `auto`.
+ */
+ quality?: 'low' | 'medium' | 'high' | 'auto';
+
+ /**
+ * The size of the generated image. One of `1024x1024`, `1024x1536`, `1536x1024`,
+ * or `auto`. Default: `auto`.
+ */
+ size?: '1024x1024' | '1024x1536' | '1536x1024' | 'auto';
+ }
+
+ export namespace ImageGeneration {
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ export interface InputImageMask {
+ /**
+ * File ID for the mask image.
+ */
+ file_id?: string;
+
+ /**
+ * Base64-encoded mask image.
+ */
+ image_url?: string;
+ }
+ }
+
+ /**
+ * A tool that allows the model to execute shell commands in a local environment.
+ */
+ export interface LocalShell {
+ /**
+ * The type of the local shell tool. Always `local_shell`.
+ */
+ type: 'local_shell';
+ }
+}
/**
* Use this option to force the model to call a specific function.
@@ -2777,8 +4262,18 @@ export interface ToolChoiceTypes {
* - `file_search`
* - `web_search_preview`
* - `computer_use_preview`
- */
- type: 'file_search' | 'web_search_preview' | 'computer_use_preview' | 'web_search_preview_2025_03_11';
+ * - `code_interpreter`
+ * - `mcp`
+ * - `image_generation`
+ */
+ type:
+ | 'file_search'
+ | 'web_search_preview'
+ | 'computer_use_preview'
+ | 'web_search_preview_2025_03_11'
+ | 'image_generation'
+ | 'code_interpreter'
+ | 'mcp';
}
/**
@@ -2788,10 +4283,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +4294,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +4313,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2860,6 +4359,12 @@ export interface ResponseCreateParamsBase {
*/
model: Shared.ResponsesModel;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* Specify additional output data to include in the model response. Currently
* supported values are:
@@ -2869,6 +4374,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
@@ -3104,6 +4614,10 @@ export declare namespace Responses {
type ResponseFunctionToolCallItem as ResponseFunctionToolCallItem,
type ResponseFunctionToolCallOutputItem as ResponseFunctionToolCallOutputItem,
type ResponseFunctionWebSearch as ResponseFunctionWebSearch,
+ type ResponseImageGenCallCompletedEvent as ResponseImageGenCallCompletedEvent,
+ type ResponseImageGenCallGeneratingEvent as ResponseImageGenCallGeneratingEvent,
+ type ResponseImageGenCallInProgressEvent as ResponseImageGenCallInProgressEvent,
+ type ResponseImageGenCallPartialImageEvent as ResponseImageGenCallPartialImageEvent,
type ResponseInProgressEvent as ResponseInProgressEvent,
type ResponseIncludable as ResponseIncludable,
type ResponseIncompleteEvent as ResponseIncompleteEvent,
@@ -3117,6 +4631,14 @@ export declare namespace Responses {
type ResponseInputMessageItem as ResponseInputMessageItem,
type ResponseInputText as ResponseInputText,
type ResponseItem as ResponseItem,
+ type ResponseMcpCallArgumentsDeltaEvent as ResponseMcpCallArgumentsDeltaEvent,
+ type ResponseMcpCallArgumentsDoneEvent as ResponseMcpCallArgumentsDoneEvent,
+ type ResponseMcpCallCompletedEvent as ResponseMcpCallCompletedEvent,
+ type ResponseMcpCallFailedEvent as ResponseMcpCallFailedEvent,
+ type ResponseMcpCallInProgressEvent as ResponseMcpCallInProgressEvent,
+ type ResponseMcpListToolsCompletedEvent as ResponseMcpListToolsCompletedEvent,
+ type ResponseMcpListToolsFailedEvent as ResponseMcpListToolsFailedEvent,
+ type ResponseMcpListToolsInProgressEvent as ResponseMcpListToolsInProgressEvent,
type ResponseOutputAudio as ResponseOutputAudio,
type ResponseOutputItem as ResponseOutputItem,
type ResponseOutputItemAddedEvent as ResponseOutputItemAddedEvent,
@@ -3124,7 +4646,13 @@ export declare namespace Responses {
type ResponseOutputMessage as ResponseOutputMessage,
type ResponseOutputRefusal as ResponseOutputRefusal,
type ResponseOutputText as ResponseOutputText,
+ type ResponseOutputTextAnnotationAddedEvent as ResponseOutputTextAnnotationAddedEvent,
+ type ResponseQueuedEvent as ResponseQueuedEvent,
+ type ResponseReasoningDeltaEvent as ResponseReasoningDeltaEvent,
+ type ResponseReasoningDoneEvent as ResponseReasoningDoneEvent,
type ResponseReasoningItem as ResponseReasoningItem,
+ type ResponseReasoningSummaryDeltaEvent as ResponseReasoningSummaryDeltaEvent,
+ type ResponseReasoningSummaryDoneEvent as ResponseReasoningSummaryDoneEvent,
type ResponseReasoningSummaryPartAddedEvent as ResponseReasoningSummaryPartAddedEvent,
type ResponseReasoningSummaryPartDoneEvent as ResponseReasoningSummaryPartDoneEvent,
type ResponseReasoningSummaryTextDeltaEvent as ResponseReasoningSummaryTextDeltaEvent,
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..ff06c96c3 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.101.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
diff --git tests/api-resources/responses/responses.test.ts tests/api-resources/responses/responses.test.ts
index cf7e9cf3c..1b2cf7fc4 100644
--- tests/api-resources/responses/responses.test.ts
+++ tests/api-resources/responses/responses.test.ts
@@ -24,6 +24,7 @@ describe('resource responses', () => {
const response = await client.responses.create({
input: 'string',
model: 'gpt-4o',
+ background: true,
include: ['file_search_call.results'],
instructions: 'instructions',
max_output_tokens: 0,
@@ -39,11 +40,11 @@ describe('resource responses', () => {
tool_choice: 'none',
tools: [
{
- type: 'file_search',
- vector_store_ids: ['string'],
- filters: { key: 'key', type: 'eq', value: 'string' },
- max_num_results: 0,
- ranking_options: { ranker: 'auto', score_threshold: 0 },
+ name: 'name',
+ parameters: { foo: 'bar' },
+ strict: true,
+ type: 'function',
+ description: 'description',
},
],
top_p: 1,
Description
This Pull Request (PR) upgrades the node version from 18 to 20 in multiple GitHub Actions workflows, adds new API methods and corresponding documentation, updates several files including package.json, and improves examples and tests.
Possible Issues
- Compatibility issues: Some dependencies or code might have compatibility problems with Node.js version 20.
- Missing checks: Ensure all code using the new methods and functionalities have corresponding validation and error handling.
- Code duplication: Some parts of the code look very similar across various methods, potentially leading to maintenance challenges.
Security Hotspots
- External Resources: The inclusion of external API calls and URLs should be monitored to ensure they are secure and not susceptible to man-in-the-middle attacks.
- Shell Command Execution: New functionality for executing shell commands locally might introduce security vulnerabilities like command injection if not properly sanitized.
Changes
Changes
Workflows:
-
ci.yml:
- node-version: '18' + node-version: '20' -
publish-jsr.yml:
- node-version: '18' + node-version: '20' -
publish-npm.yml:
- node-version: '18' + node-version: '20'
Configuration:
-
.release-please-manifest.json:
- Updated version from
4.96.2to4.101.0.
- Updated version from
-
.stats.yml:
- Increased
configured_endpointsfrom 97 to 101.
- Increased
Documentation:
-
CHANGELOG.md:
- Added entries for versions
4.97.0to4.101.0with new features, API updates, and document improvements.
- Added entries for versions
-
README.md, SECURITY.md, api.md, realtime.md:
- Typographical corrections and added detailed examples for new API methods.
Source Code:
-
New Files:
-
src/resources/fine-tuning/alpha.ts -
src/resources/fine-tuning/alpha/graders.ts -
src/resources/fine-tuning/alpha/index.ts -
src/resources/fine-tuning/methods.ts -
src/resources/graders.ts -
src/resources/graders/grader-models.ts -
src/resources/graders/graders.ts -
src/resources/graders/index.ts
-
-
Modified Files:
-
src/index.ts, src/resources/evals/evals.ts, src/resources/fine-tuning/alpha/alpha.ts, src/resources/fine-tuning/alpha/graders.ts, src/resources/fine-tuning/checkpoints/permissions.ts, src/resources/responses/responses.tsetc. throughout the project files including updated Types and Methods.
-
-
Examples:
- Update examples/azure/realtime/websocket.ts, ws.ts, and other example scripts to fix typographical errors (
procesingtoprocessing).
- Update examples/azure/realtime/websocket.ts, ws.ts, and other example scripts to fix typographical errors (
-
Tests:
- Added and updated tests across various files such as
tests/api-resources/fine-tuning/jobs/jobs.test.tsto include tests for new methods and parameter variations.
- Added and updated tests across various files such as
Versioning:
- export const VERSION = '4.96.2'; // x-release-please-version
+ export const VERSION = '4.101.0'; // x-release-please-version
sequenceDiagram
participant Developer
participant GitHub Actions
participant Node.js
participant API Resources
Developer->>GitHub Actions: Push code
GitHub Actions->>Node.js: Set up Node Version 20
GitHub Actions->>API Resources: Add/Update methods and tests
Developer->>API Resources: Use New API Methods
Overall Recommendations
- Ensure compatibility with Node.js 20 for all dependencies and functionalities.
- Verify that new methods and functionalities undergo comprehensive testing.
- Sanitize and validate any data inputs considered for shell commands or remote executions to preempt potential security vulnerabilities.
bedrock debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..514215a1e 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.101.0"
}
diff --git .stats.yml .stats.yml
index d92408173..4b4f19c91 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 101
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-a5651cb97f86d1e2531af6aef8c5230f1ea350560fbae790ca2e481b30a6c217.yml
+openapi_spec_hash: 66a5104fd3bb43383cf919225df7a6fd
+config_hash: bb657c3fed232a56930035de3aaed936
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..1e427d912 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,80 @@
# Changelog
+## 4.101.0 (2025-05-21)
+
+Full Changelog: [v4.100.0...v4.101.0](https://github.com/openai/openai-node/compare/v4.100.0...v4.101.0)
+
+### Features
+
+* **api:** new API tools ([fb4014f](https://github.com/openai/openai-node/commit/fb4014ffac7b220d37bd03c94fa745386b010bf0))
+
+
+### Chores
+
+* **docs:** grammar improvements ([7761cfb](https://github.com/openai/openai-node/commit/7761cfb0a8a56d056a73c046a6a613f66ada4abe))
+* **internal:** version bump ([b40e830](https://github.com/openai/openai-node/commit/b40e8302ec11683b6a360a050236ad1284afc760))
+
+## 4.100.0 (2025-05-16)
+
+Full Changelog: [v4.99.0...v4.100.0](https://github.com/openai/openai-node/compare/v4.99.0...v4.100.0)
+
+### Features
+
+* **api:** further updates for evals API ([3f6f248](https://github.com/openai/openai-node/commit/3f6f248191b45015924be76fd5154d149c4ed8a0))
+
+
+### Chores
+
+* **internal:** version bump ([5123fe0](https://github.com/openai/openai-node/commit/5123fe08a56f3d0040b1cc67129382f3eacc3cca))
+
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git SECURITY.md SECURITY.md
index 3b3bd8a66..4adb0c54f 100644
--- SECURITY.md
+++ SECURITY.md
@@ -16,13 +16,13 @@ before making any information public.
## Reporting Non-SDK Related Security Issues
If you encounter security issues that are not directly related to SDKs but pertain to the services
-or products provided by OpenAI please follow the respective company's security reporting guidelines.
+or products provided by OpenAI, please follow the respective company's security reporting guidelines.
### OpenAI Terms and Policies
Our Security Policy can be found at [Security Policy URL](https://openai.com/policies/coordinated-vulnerability-disclosure-policy).
-Please contact [email protected] for any questions or concerns regarding security of our services.
+Please contact [email protected] for any questions or concerns regarding the security of our services.
---
diff --git api.md api.md
index 49e6548a8..f7fb47e3e 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -605,6 +645,10 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionWebSearch</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallGeneratingEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallPartialImageEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncludable</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncompleteEvent</a></code>
@@ -618,6 +662,14 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseInputMessageItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInputText</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDoneEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputAudio</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItemAddedEvent</a></code>
@@ -625,7 +677,13 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputMessage</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputRefusal</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputText</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseOutputTextAnnotationAddedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseQueuedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartAddedEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryTextDeltaEvent</a></code>
@@ -669,10 +727,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..42bf0a235 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.101.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..e8cd7277c 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.101.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..537c18f43 100644
--- src/index.ts
+++ src/index.ts
@@ -71,19 +71,17 @@ import {
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,6 +303,7 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
@@ -366,6 +365,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -487,6 +487,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +533,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/realtime.ts src/resources/beta/realtime/realtime.ts
index 5012b1edd..26fba883e 100644
--- src/resources/beta/realtime/realtime.ts
+++ src/resources/beta/realtime/realtime.ts
@@ -2571,7 +2571,7 @@ export interface TranscriptionSessionUpdatedEvent {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
session: TranscriptionSessionsAPI.TranscriptionSession;
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..83e8c47ad 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
@@ -29,7 +35,7 @@ export class TranscriptionSessions extends APIResource {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
export interface TranscriptionSession {
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreateParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..05a656619 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -27,7 +28,8 @@ export class Evals extends APIResource {
/**
* Create the structure of an evaluation that can be used to test a model's
- * performance. An evaluation is a set of testing criteria and a datasource. After
+ * performance. An evaluation is a set of testing criteria and the config for a
+ * data source, which dictates the schema of the data used in the evaluation. After
* creating an evaluation, you can run it on different models and model parameters.
* We support several types of graders and datasources. For more information, see
* the [Evals guide](https://platform.openai.com/docs/guides/evals).
@@ -104,88 +106,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -210,90 +131,13 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalCreateResponse {
/**
@@ -309,7 +153,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +182,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +244,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -465,7 +258,7 @@ export namespace EvalCreateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalRetrieveResponse {
/**
@@ -481,7 +274,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +303,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +365,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -637,7 +379,7 @@ export namespace EvalRetrieveResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalUpdateResponse {
/**
@@ -653,7 +395,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +424,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +486,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
*, The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -809,7 +500,7 @@ export namespace EvalUpdateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalListResponse {
/**
@@ -825,7 +516,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +545,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +607,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -985,17 +625,21 @@ export interface EvalDeleteResponse {
export interface EvalCreateParams {
/**
- * The configuration for the data source used for the evaluation runs.
+ * The configuration for the data source used for the evaluation runs. Dictates the
+ * schema of the data used in the evaluation.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
- * A list of graders for all eval runs in this group.
+ * A list of graders for all eval runs in this group. Graders can reference
+ * variables in the data source using double curly braces notation, like
+ * `{{item.variable_name}}`. To reference the model's output, use the `sample`
+ * namespace (ie, `{{sample.output_text}}`).
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +688,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +703,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored_completions`.
+ */
+ type: 'stored_completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1067,7 +725,7 @@ export namespace EvalCreateParams {
export interface LabelModel {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
input: Array<LabelModel.SimpleInputMessage | LabelModel.EvalItem>;
@@ -1154,29 +812,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +834,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +880,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..e761e2160 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -19,7 +19,9 @@ export class Runs extends APIResource {
outputItems: OutputItemsAPI.OutputItems = new OutputItemsAPI.OutputItems(this._client);
/**
- * Create a new evaluation run. This is the endpoint that will kick off grading.
+ * Kicks off a new run for a given evaluation, specifying the data source, and what
+ * model configuration to use to test. The datasource will be validated against the
+ * schema specified in the config of the evaluation.
*/
create(
evalId: string,
@@ -85,7 +87,7 @@ export class RunListResponsesPage extends CursorPage<RunListResponse> {}
*/
export interface CreateEvalCompletionsRunDataSource {
/**
- * A StoredCompletionsRunDataSource configuration describing a set of filters
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalCompletionsRunDataSource.FileContent
@@ -97,6 +99,12 @@ export interface CreateEvalCompletionsRunDataSource {
*/
type: 'completions';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalCompletionsRunDataSource.Template
| CreateEvalCompletionsRunDataSource.ItemReference;
@@ -185,7 +193,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<ResponsesAPI.EasyInputMessage | Template.Message>;
@@ -241,7 +249,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"
*/
item_reference: string;
@@ -279,6 +287,9 @@ export namespace CreateEvalCompletionsRunDataSource {
* eval
*/
export interface CreateEvalJSONLRunDataSource {
+ /**
+ * Determines what populates the `item` namespace in the data source.
+ */
source: CreateEvalJSONLRunDataSource.FileContent | CreateEvalJSONLRunDataSource.FileID;
/**
@@ -356,7 +367,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +434,34 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +503,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -505,14 +516,8 @@ export namespace RunCreateResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +544,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -553,7 +563,7 @@ export namespace RunCreateResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -621,7 +631,7 @@ export namespace RunCreateResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -749,7 +759,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +826,34 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +895,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -898,14 +908,8 @@ export namespace RunRetrieveResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +936,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -946,7 +955,7 @@ export namespace RunRetrieveResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1014,7 +1023,7 @@ export namespace RunRetrieveResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1139,10 +1148,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1215,34 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1284,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1291,14 +1297,8 @@ export namespace RunListResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1325,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1339,7 +1344,7 @@ export namespace RunListResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1407,7 +1412,7 @@ export namespace RunListResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1543,7 +1548,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1615,34 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1684,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1692,14 +1697,8 @@ export namespace RunCancelResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1725,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1740,7 +1744,7 @@ export namespace RunCancelResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1808,7 +1812,7 @@ export namespace RunCancelResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1947,7 +1951,7 @@ export namespace RunCreateParams {
*/
export interface CreateEvalResponsesRunDataSource {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalResponsesRunDataSource.FileContent
@@ -1955,10 +1959,16 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalResponsesRunDataSource.Template
| CreateEvalResponsesRunDataSource.ItemReference;
@@ -2013,12 +2023,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2032,14 +2036,8 @@ export namespace RunCreateParams {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2064,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -2080,7 +2083,7 @@ export namespace RunCreateParams {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -2148,7 +2151,7 @@ export namespace RunCreateParams {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparameters,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | Tex,tSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..9d827615c 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -33,10 +33,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +54,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..fdf51f224 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -37,7 +37,13 @@ export type ParsedResponseOutputItem<ParsedT> =
| ResponseFileSearchToolCall
| ResponseFunctionWebSearch
| ResponseComputerToolCall
- | ResponseReasoningItem;
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
export interface ParsedResponse<ParsedT> extends Response {
output: Array<ParsedResponseOutputItem<ParsedT>>;
@@ -61,6 +67,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +104,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +131,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -158,7 +186,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +237,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +286,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -392,6 +420,12 @@ export interface Response {
*/
top_p: number | null;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* An upper bound for the number of tokens that can be generated for a response,
* including visible output tokens and
@@ -437,7 +471,7 @@ export interface Response {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
status?: ResponseStatus;
@@ -665,6 +699,11 @@ export interface ResponseCodeInterpreterToolCall {
* The type of the code interpreter tool call. Always `code_interpreter_call`.
*/
type: 'code_interpreter_call';
+
+ /**
+ * The ID of the container used to run the code.
+ */
+ container_id?: string;
}
export namespace ResponseCodeInterpreterToolCall {
@@ -1557,6 +1596,114 @@ export interface ResponseFunctionWebSearch {
type: 'web_search_call';
}
+/**
+ * Emitted when an image generation tool call has completed and the final image is
+ * available.
+ */
+export interface ResponseImageGenCallCompletedEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.completed'.
+ */
+ type: 'response.image_generation_call.completed';
+}
+
+/**
+ * Emitted when an image generation tool call is actively generating an image
+ * (intermediate state).
+ */
+export interface ResponseImageGenCallGeneratingEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.generating'.
+ */
+ type: 'response.image_generation_call.generating';
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number?: number;
+}
+
+/**
+ * Emitted when an image generation tool call is in progress.
+ */
+export interface ResponseImageGenCallInProgressEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.in_progress'.
+ */
+ type: 'response.image_generation_call.in_progress';
+}
+
+/**
+ * Emitted when a partial image is available during image generation streaming.
+ */
+export interface ResponseImageGenCallPartialImageEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * Base64-encoded partial image data, suitable for rendering as an image.
+ */
+ partial_image_b64: string;
+
+ /**
+ * 0-based index for the partial image (backend is 1-based, but this is 0-based for
+ * the user).
+ */
+ partial_image_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.partial_image'.
+ */
+ type: 'response.image_generation_call.partial_image';
+}
+
/**
* Emitted when the response is in progress.
*/
@@ -1581,11 +1728,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1650,7 +1803,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1820,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1704,6 +1857,14 @@ export type ResponseInputItem =
| ResponseFunctionToolCall
| ResponseInputItem.FunctionCallOutput
| ResponseReasoningItem
+ | ResponseInputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseInputItem.LocalShellCall
+ | ResponseInputItem.LocalShellCallOutput
+ | ResponseInputItem.McpListTools
+ | ResponseInputItem.McpApprovalRequest
+ | ResponseInputItem.McpApprovalResponse
+ | ResponseInputItem.McpCall
| ResponseInputItem.ItemReference;
export namespace ResponseInputItem {
@@ -1758,19 +1919,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +1947,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,172 +1979,1079 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
- * An internal identifier for an item to reference.
+ * An image generation request made by the model.
*/
- export interface ItemReference {
+ export interface ImageGenerationCall {
/**
- * The ID of the item to reference.
+ * The unique ID of the image generation call.
*/
id: string;
/**
- * The type of item to reference. Always `item_reference`.
+ * The generated image encoded in base64.
*/
- type: 'item_reference';
- }
-}
+ result: string | null;
-/**
- * A list of one or many input items to the model, containing different content
- * types.
- */
-export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
-export interface ResponseInputMessageItem {
- /**
- * The unique ID of the message input.
- */
- id: string;
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
/**
- * A list of one or many input items to the model, containing different content
- * types.
+ * A tool call to run a command on the local shell.
*/
- content: ResponseInputMessageContentList;
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
- /**
- * The role of the message input. One of `user`, `system`, or `developer`.
- */
- role: 'user' | 'system' | 'developer';
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
- /**
- * The status of item. One of `in_progress`, `completed`, or `incomplete`.
- * Populated when items are returned via API.
- */
- status?: 'in_progress' | 'completed' | 'incomplete';
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
- /**
- * The type of the message input. Always set to `message`.
- */
- type?: 'message';
-}
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
-/**
- * A text input to the model.
- */
-export interface ResponseInputText {
- /**
- * The text input to the model.
- */
- text: string;
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
- /**
- * The type of the input item. Always `input_text`.
- */
- type: 'input_text';
-}
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
-/**
- * Content item used to generate a response.
- */
-export type ResponseItem =
- | ResponseInputMessageItem
- | ResponseOutputMessage
- | ResponseFileSearchToolCall
- | ResponseComputerToolCall
- | ResponseComputerToolCallOutputItem
- | ResponseFunctionWebSearch
- | ResponseFunctionToolCallItem
- | ResponseFunctionToolCallOutputItem;
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
-/**
- * An audio output from the model.
- */
-export interface ResponseOutputAudio {
- /**
- * Base64-encoded audio data from the model.
- */
- data: string;
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
- /**
- * The transcript of the audio data from the model.
- */
- transcript: string;
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
- /**
- * The type of the output audio. Always `output_audio`.
- */
- type: 'output_audio';
-}
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
-/**
- * An output message from the model.
- */
-export type ResponseOutputItem =
- | ResponseOutputMessage
- | ResponseFileSearchToolCall
- | ResponseFunctionToolCall
- | ResponseFunctionWebSearch
- | ResponseComputerToolCall
- | ResponseReasoningItem;
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
-/**
- * Emitted when a new output item is added.
- */
-export interface ResponseOutputItemAddedEvent {
/**
- * The output item that was added.
+ * The output of a local shell tool call.
*/
- item: ResponseOutputItem;
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
- /**
- * The index of the output item that was added.
- */
- output_index: number;
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
- /**
- * The type of the event. Always `response.output_item.added`.
- */
- type: 'response.output_item.added';
-}
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
-/**
- * Emitted when an output item is marked done.
- */
-export interface ResponseOutputItemDoneEvent {
- /**
- * The output item that was marked done.
- */
- item: ResponseOutputItem;
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
/**
- * The index of the output item that was marked done.
+ * A list of tools available on an MCP server.
*/
- output_index: number;
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
- /**
- * The type of the event. Always `response.output_item.done`.
- */
- type: 'response.output_item.done';
-}
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
-/**
- * An output message from the model.
- */
-export interface ResponseOutputMessage {
- /**
- * The unique ID of the output message.
- */
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * The unique ID of the approval response
+ */
+ id?: string | null;
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * An internal identifier for an item to reference.
+ */
+ export interface ItemReference {
+ /**
+ * The ID of the item to reference.
+ */
+ id: string;
+
+ /**
+ * The type of item to reference. Always `item_reference`.
+ */
+ type?: 'item_reference' | null;
+ }
+}
+
+/**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+
+export interface ResponseInputMessageItem {
+ /**
+ * The unique ID of the message input.
+ */
+ id: string;
+
+ /**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+ content: ResponseInputMessageContentList;
+
+ /**
+ * The role of the message input. One of `user`, `system`, or `developer`.
+ */
+ role: 'user' | 'system' | 'developer';
+
+ /**
+ * The status of item. One of `in_progress`, `completed`, or `incomplete`.
+ * Populated when items are returned via API.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the message input. Always set to `message`.
+ */
+ type?: 'message';
+}
+
+/**
+ * A text input to the model.
+ */
+export interface ResponseInputText {
+ /**
+ * The text input to the model.
+ */
+ text: string;
+
+ /**
+ * The type of the input item. Always `input_text`.
+ */
+ type: 'input_text';
+}
+
+/**
+ * Content item used to generate a response.
+ */
+export type ResponseItem =
+ | ResponseInputMessageItem
+ | ResponseOutputMessage
+ | ResponseFileSearchToolCall
+ | ResponseComputerToolCall
+ | ResponseComputerToolCallOutputItem
+ | ResponseFunctionWebSearch
+ | ResponseFunctionToolCallItem
+ | ResponseFunctionToolCallOutputItem
+ | ResponseItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseItem.LocalShellCall
+ | ResponseItem.LocalShellCallOutput
+ | ResponseItem.McpListTools
+ | ResponseItem.McpApprovalRequest
+ | ResponseItem.McpApprovalResponse
+ | ResponseItem.McpCall;
+
+export namespace ResponseItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * The output of a local shell tool call.
+ */
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
+
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
+
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The unique ID of the approval response
+ */
+ id: string;
+
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the arguments of an MCP tool
+ * call.
+ */
+export interface ResponseMcpCallArgumentsDeltaEvent {
+ /**
+ * The partial update to the arguments for the MCP tool call.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_delta'.
+ */
+ type: 'response.mcp_call.arguments_delta';
+}
+
+/**
+ * Emitted when the arguments for an MCP tool call are finalized.
+ */
+export interface ResponseMcpCallArgumentsDoneEvent {
+ /**
+ * The finalized arguments for the MCP tool call.
+ */
+ arguments: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_done'.
+ */
+ type: 'response.mcp_call.arguments_done';
+}
+
+/**
+ * Emitted when an MCP tool call has completed successfully.
+ */
+export interface ResponseMcpCallCompletedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_call.completed'.
+ */
+ type: 'response.mcp_call.completed';
+}
+
+/**
+ * Emitted when an MCP tool call has failed.
+ */
+export interface ResponseMcpCallFailedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_call.failed'.
+ */
+ type: 'response.mcp_call.failed';
+}
+
+/**
+ * Emitted when an MCP tool call is in progress.
+ */
+export interface ResponseMcpCallInProgressEvent {
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.in_progress'.
+ */
+ type: 'response.mcp_call.in_progress';
+}
+
+/**
+ * Emitted when the list of available MCP tools has been successfully retrieved.
+ */
+export interface ResponseMcpListToolsCompletedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.completed'.
+ */
+ type: 'response.mcp_list_tools.completed';
+}
+
+/**
+ * Emitted when the attempt to list available MCP tools has failed.
+ */
+export interface ResponseMcpListToolsFailedEvent {
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.failed'.
+ */
+ type: 'response.mcp_list_tools.failed';
+}
+
+/**
+ * Emitted when the system is in the process of retrieving the list of available
+ * MCP tools.
+ */
+export interface ResponseMcpListToolsInProgressEvent {
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.in_progress'.
+ */
+ type: 'response.mcp_list_tools.in_progress';
+}
+
+/**
+ * An audio output from the model.
+ */
+export interface ResponseOutputAudio {
+ /**
+ * Base64-encoded audio data from the model.
+ */
+ data: string;
+
+ /**
+ * The transcript of the audio data from the model.
+ */
+ transcript: string;
+
+ /**
+ * The type of the output audio. Always `output_audio`.
+ */
+ type: 'output_audio';
+}
+
+/**
+ * An output message from the model.
+ */
+export type ResponseOutputItem =
+ | ResponseOutputMessage
+ | ResponseFileSearchToolCall
+ | ResponseFunctionToolCall
+ | ResponseFunctionWebSearch
+ | ResponseComputerToolCall
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
+
+export namespace ResponseOutputItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+}
+
+/**
+ * Emitted when a new output item is added.
+ */
+export interface ResponseOutputItemAddedEvent {
+ /**
+ * The output item that was added.
+ */
+ item: ResponseOutputItem;
+
+ /**
+ * The index of the output item that was added.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always `response.output_item.added`.
+ */
+ type: 'response.output_item.added';
+}
+
+/**
+ * Emitted when an output item is marked done.
+ */
+export interface ResponseOutputItemDoneEvent {
+ /**
+ * The output item that was marked done.
+ */
+ item: ResponseOutputItem;
+
+ /**
+ * The index of the output item that was marked done.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always `response.output_item.done`.
+ */
+ type: 'response.output_item.done';
+}
+
+/**
+ * An output message from the model.
+ */
+export interface ResponseOutputMessage {
+ /**
+ * The unique ID of the output message.
+ */
id: string;
/**
@@ -2117,9 +3185,121 @@ export namespace ResponseOutputText {
}
}
+/**
+ * Emitted when an annotation is added to output text content.
+ */
+export interface ResponseOutputTextAnnotationAddedEvent {
+ /**
+ * The annotation object being added. (See annotation schema for details.)
+ */
+ annotation: unknown;
+
+ /**
+ * The index of the annotation within the content part.
+ */
+ annotation_index: number;
+
+ /**
+ * The index of the content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The unique identifier of the item to which the annotation is being added.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.output_text_annotation.added'.
+ */
+ type: 'response.output_text_annotation.added';
+}
+
+/**
+ * Emitted when a response is queued and waiting to be processed.
+ */
+export interface ResponseQueuedEvent {
+ /**
+ * The full response object that is queued.
+ */
+ response: Response;
+
+ /**
+ * The type of the event. Always 'response.queued'.
+ */
+ type: 'response.queued';
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the reasoning content.
+ */
+export interface ResponseReasoningDeltaEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The partial update to the reasoning content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which reasoning is being updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning.delta'.
+ */
+ type: 'response.reasoning.delta';
+}
+
+/**
+ * Emitted when the reasoning content is finalized for an item.
+ */
+export interface ResponseReasoningDoneEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The unique identifier of the item for which reasoning is finalized.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The finalized reasoning text.
+ */
+ text: string;
+
+ /**
+ * The type of the event. Always 'response.reasoning.done'.
+ */
+ type: 'response.reasoning.done';
+}
+
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +3317,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2158,6 +3344,67 @@ export namespace ResponseReasoningItem {
}
}
+/**
+ * Emitted when there is a delta (partial update) to the reasoning summary content.
+ */
+export interface ResponseReasoningSummaryDeltaEvent {
+ /**
+ * The partial update to the reasoning summary content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which the reasoning summary is being
+ * updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.delta'.
+ */
+ type: 'response.reasoning_summary.delta';
+}
+
+/**
+ * Emitted when the reasoning summary content is finalized for an item.
+ */
+export interface ResponseReasoningSummaryDoneEvent {
+ /**
+ * The unique identifier of the item for which the reasoning summary is finalized.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The finalized reasoning summary text.
+ */
+ text: string;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.done'.
+ */
+ type: 'response.reasoning_summary.done';
+}
+
/**
* Emitted when a new reasoning summary part is added.
*/
@@ -2374,9 +3621,9 @@ export interface ResponseRefusalDoneEvent {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
-export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'incomplete';
+export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'cancelled' | 'queued' | 'incomplete';
,/**
* Emitted when there is a partial audio response.
@@ -2417,7 +3664,25 @@ export type ResponseStreamEvent =
| ResponseTextDoneEvent
| ResponseWebSearchCallCompletedEvent
| ResponseWebSearchCallInProgressEvent
- | ResponseWebSearchCallSearchingEvent;
+ | ResponseWebSearchCallSearchingEvent
+ | ResponseImageGenCallCompletedEvent
+ | ResponseImageGenCallGeneratingEvent
+ | ResponseImageGenCallInProgressEvent
+ | ResponseImageGenCallPartialImageEvent
+ | ResponseMcpCallArgumentsDeltaEvent
+ | ResponseMcpCallArgumentsDoneEvent
+ | ResponseMcpCallCompletedEvent
+ | ResponseMcpCallFailedEvent
+ | ResponseMcpCallInProgressEvent
+ | ResponseMcpListToolsCompletedEvent
+ | ResponseMcpListToolsFailedEvent
+ | ResponseMcpListToolsInProgressEvent
+ | ResponseOutputTextAnnotationAddedEvent
+ | ResponseQueuedEvent
+ | ResponseReasoningDeltaEvent
+ | ResponseReasoningDoneEvent
+ | ResponseReasoningSummaryDeltaEvent
+ | ResponseReasoningSummaryDoneEvent;
/**
* Emitted when a text annotation is added.
@@ -2730,11 +3995,231 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool =
+ | FunctionTool
+ | FileSearchTool
+ | WebSearchTool
+ | ComputerTool
+ | Tool.Mcp
+ | Tool.CodeInterpreter
+ | Tool.ImageGeneration
+ | Tool.LocalShell;
+
+export namespace Tool {
+ /**
+ * Give the model access to additional tools via remote Model Context Protocol
+ * (MCP) servers.
+ * [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp).
+ */
+ export interface Mcp {
+ /**
+ * A label for this MCP server, used to identify it in tool calls.
+ */
+ server_label: string;
+
+ /**
+ * The URL for the MCP server.
+ */
+ server_url: string;
+
+ /**
+ * The type of the MCP tool. Always `mcp`.
+ */
+ type: 'mcp';
+
+ /**
+ * List of allowed tool names or a filter object.
+ */
+ allowed_tools?: Array<string> | Mcp.McpAllowedToolsFilter | null;
+
+ /**
+ * Optional HTTP headers to send to the MCP server. Use for authentication or other
+ * purposes.
+ */
+ headers?: Record<string, string> | null;
+
+ /**
+ * Specify which of the MCP server's tools require approval.
+ */
+ require_approval?: Mcp.McpToolApprovalFilter | 'always' | 'never' | null;
+ }
+
+ export namespace Mcp {
+ /**
+ * A filter object to specify which tools are allowed.
+ */
+ export interface McpAllowedToolsFilter {
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export interface McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ always?: McpToolApprovalFilter.Always;
+
+ /**
+ * A list of tools that never require approval.
+ */
+ never?: McpToolApprovalFilter.Never;
+
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export namespace McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ export interface Always {
+ /**
+ * List of tools that require approval.
+ */
+ tool_names?: Array<string>;
+ }
+
+ /**
+ * A list of tools that never require approval.
+ */
+ export interface Never {
+ /**
+ * List of tools that do not require approval.
+ */
+ tool_names?: Array<string>;
+ }
+ }
+ }
+
+ /**
+ * A tool that runs Python code to help generate a response to a prompt.
+ */
+ export interface CodeInterpreter {
+ /**
+ * The code interpreter container. Can be a container ID or an object that
+ * specifies uploaded file IDs to make available to your code.
+ */
+ container: string | CodeInterpreter.CodeInterpreterToolAuto;
+
+ /**
+ * The type of the code interpreter tool. Always `code_interpreter`.
+ */
+ type: 'code_interpreter';
+ }
+
+ export namespace CodeInterpreter {
+ /**
+ * Configuration for a code interpreter container. Optionally specify the IDs of
+ * the files to run the code on.
+ */
+ export interface CodeInterpreterToolAuto {
+ /**
+ * Always `auto`.
+ */
+ type: 'auto';
+
+ /**
+ * An optional list of uploaded files to make available to your code.
+ */
+ file_ids?: Array<string>;
+ }
+ }
+
+ /**
+ * A tool that generates images using a model like `gpt-image-1`.
+ */
+ export interface ImageGeneration {
+ /**
+ * The type of the image generation tool. Always `image_generation`.
+ */
+ type: 'image_generation';
+
+ /**
+ * Background type for the generated image. One of `transparent`, `opaque`, or
+ * `auto`. Default: `auto`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto';
+
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ input_image_mask?: ImageGeneration.InputImageMask;
+
+ /**
+ * The image generation model to use. Default: `gpt-image-1`.
+ */
+ model?: 'gpt-image-1';
+
+ /**
+ * Moderation level for the generated image. Default: `auto`.
+ */
+ moderation?: 'auto' | 'low';
+
+ /**
+ * Compression level for the output image. Default: 100.
+ */
+ output_compression?: number;
+
+ /**
+ * The output format of the generated image. One of `png`, `webp`, or `jpeg`.
+ * Default: `png`.
+ */
+ output_format?: 'png' | 'webp' | 'jpeg';
+
+ /**
+ * Number of partial images to generate in streaming mode, from 0 (default value)
+ * to 3.
+ */
+ partial_images?: number;
+
+ /**
+ * The quality of the generated image. One of `low`, `medium`, `high`, or `auto`.
+ * Default: `auto`.
+ */
+ quality?: 'low' | 'medium' | 'high' | 'auto';
+
+ /**
+ * The size of the generated image. One of `1024x1024`, `1024x1536`, `1536x1024`,
+ * or `auto`. Default: `auto`.
+ */
+ size?: '1024x1024' | '1024x1536' | '1536x1024' | 'auto';
+ }
+
+ export namespace ImageGeneration {
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ export interface InputImageMask {
+ /**
+ * File ID for the mask image.
+ */
+ file_id?: string;
+
+ /**
+ * Base64-encoded mask image.
+ */
+ image_url?: string;
+ }
+ }
+
+ /**
+ * A tool that allows the model to execute shell commands in a local environment.
+ */
+ export interface LocalShell {
+ /**
+ * The type of the local shell tool. Always `local_shell`.
+ */
+ type: 'local_shell';
+ }
+}
/**
* Use this option to force the model to call a specific function.
@@ -2777,8 +4262,18 @@ export interface ToolChoiceTypes {
* - `file_search`
* - `web_search_preview`
* - `computer_use_preview`
- */
- type: 'file_search' | 'web_search_preview' | 'computer_use_preview' | 'web_search_preview_2025_03_11';
+ * - `code_interpreter`
+ * - `mcp`
+ * - `image_generation`
+ */
+ type:
+ | 'file_search'
+ | 'web_search_preview'
+ | 'computer_use_preview'
+ | 'web_search_preview_2025_03_11'
+ | 'image_generation'
+ | 'code_interpreter'
+ | 'mcp';
}
/**
@@ -2788,10 +4283,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +4294,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +4313,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2860,6 +4359,12 @@ export interface ResponseCreateParamsBase {
*/
model: Shared.ResponsesModel;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* Specify additional output data to include in the model response. Currently
* supported values are:
@@ -2869,6 +4374,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
@@ -3104,6 +4614,10 @@ export declare namespace Responses {
type ResponseFunctionToolCallItem as ResponseFunctionToolCallItem,
type ResponseFunctionToolCallOutputItem as ResponseFunctionToolCallOutputItem,
type ResponseFunctionWebSearch as ResponseFunctionWebSearch,
+ type ResponseImageGenCallCompletedEvent as ResponseImageGenCallCompletedEvent,
+ type ResponseImageGenCallGeneratingEvent as ResponseImageGenCallGeneratingEvent,
+ type ResponseImageGenCallInProgressEvent as ResponseImageGenCallInProgressEvent,
+ type ResponseImageGenCallPartialImageEvent as ResponseImageGenCallPartialImageEvent,
type ResponseInProgressEvent as ResponseInProgressEvent,
type ResponseIncludable as ResponseIncludable,
type ResponseIncompleteEvent as ResponseIncompleteEvent,
@@ -3117,6 +4631,14 @@ export declare namespace Responses {
type ResponseInputMessageItem as ResponseInputMessageItem,
type ResponseInputText as ResponseInputText,
type ResponseItem as ResponseItem,
+ type ResponseMcpCallArgumentsDeltaEvent as ResponseMcpCallArgumentsDeltaEvent,
+ type ResponseMcpCallArgumentsDoneEvent as ResponseMcpCallArgumentsDoneEvent,
+ type ResponseMcpCallCompletedEvent as ResponseMcpCallCompletedEvent,
+ type ResponseMcpCallFailedEvent as ResponseMcpCallFailedEvent,
+ type ResponseMcpCallInProgressEvent as ResponseMcpCallInProgressEvent,
+ type ResponseMcpListToolsCompletedEvent as ResponseMcpListToolsCompletedEvent,
+ type ResponseMcpListToolsFailedEvent as ResponseMcpListToolsFailedEvent,
+ type ResponseMcpListToolsInProgressEvent as ResponseMcpListToolsInProgressEvent,
type ResponseOutputAudio as ResponseOutputAudio,
type ResponseOutputItem as ResponseOutputItem,
type ResponseOutputItemAddedEvent as ResponseOutputItemAddedEvent,
@@ -3124,7 +4646,13 @@ export declare namespace Responses {
type Response,OutputMessage as ResponseOutputMessage,
type ResponseOutputRefusal as ResponseOutputRefusal,
type ResponseOutputText as ResponseOutputText,
+ type ResponseOutputTextAnnotationAddedEvent as ResponseOutputTextAnnotationAddedEvent,
+ type ResponseQueuedEvent as ResponseQueuedEvent,
+ type ResponseReasoningDeltaEvent as ResponseReasoningDeltaEvent,
+ type ResponseReasoningDoneEvent as ResponseReasoningDoneEvent,
type ResponseReasoningItem as ResponseReasoningItem,
+ type ResponseReasoningSummaryDeltaEvent as ResponseReasoningSummaryDeltaEvent,
+ type ResponseReasoningSummaryDoneEvent as ResponseReasoningSummaryDoneEvent,
type ResponseReasoningSummaryPartAddedEvent as ResponseReasoningSummaryPartAddedEvent,
type ResponseReasoningSummaryPartDoneEvent as ResponseReasoningSummaryPartDoneEvent,
type ResponseReasoningSummaryTextDeltaEvent as ResponseReasoningSummaryTextDeltaEvent,
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..ff06c96c3 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.101.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responseProm,ise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resourc,e images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
diff --git tests/api-resources/responses/responses.test.ts tests/api-resources/responses/responses.test.ts
index cf7e9cf3c..1b2cf7fc4 100644
--- tests/api-resources/responses/responses.test.ts
+++ tests/api-resources/responses/responses.test.ts
@@ -24,6 +24,7 @@ describe('resource responses', () => {
const response = await client.responses.create({
input: 'string',
model: 'gpt-4o',
+ background: true,
include: ['file_search_call.results'],
instructions: 'instructions',
max_output_tokens: 0,
@@ -39,11 +40,11 @@ describe('resource responses', () => {
tool_choice: 'none',
tools: [
{
- type: 'file_search',
- vector_store_ids: ['string'],
- filters: { key: 'key', type: 'eq', value: 'string' },
- max_num_results: 0,
- ranking_options: { ranker: 'auto', score_threshold: 0 },
+ name: 'name',
+ parameters: { foo: 'bar' },
+ strict: true,
+ type: 'function',
+ description: 'description',
},
],
top_p: 1,
Description
This pull request upgrades the Node.js version from 18 to 20 in CI workflows and includes several new OpenAI API features and improvements. The PR contains version updates from 4.96.2 to 4.101.0, updates to the API definitions to support new features like reinforcement fine-tuning, MCP (Model Context Protocol), and image generation tools, and fixes typos in documentation.
Changes
Changes
CI Workflows Configuration
- Updated Node.js version from 18 to 20 in the CI workflow files
- Updated in
.github/workflows/ci.yml - Updated in
.github/workflows/publish-jsr.yml - Updated in
.github/workflows/publish-npm.yml
- Updated in
Version Updates
- Updated version from 4.96.2 to 4.101.0 in:
-
.release-please-manifest.json -
jsr.json -
package.json -
src/version.ts
-
Documentation and Examples
- Fixed typos in
README.mdandSECURITY.md - Fixed "procesing" → "processing" typos in examples
- Added JSDoc examples to many API methods
- Improved formatting in
examples/tsconfig.json
API Updates
-
Fine-Tuning API:
- Added reinforcement fine-tuning API support
- Added pause/resume functionality for fine-tuning jobs
- Added new graders module for evaluation
-
Responses API:
- Added numerous new event types and response formats
- Added MCP (Model Context Protocol) support
- Added image generation capabilities
-
Image Generation:
- Added new image sizes options
- Added background transparency options
-
Evals API:
- Updated schemas for evaluation tools
-
Audio API:
- Added new chunking strategy options for transcriptions
Stats Updates
- Updated
.stats.ymlwith new metrics (endpoints and hashes)
sequenceDiagram
participant GitHub as GitHub Actions
participant NPM as NPM Registry
participant JSR as JSR Registry
participant Users as SDK Users
Note over GitHub: Update Node.js version from 18 to 20
GitHub->>GitHub: Update CI workflows
GitHub->>GitHub: Run tests on Node.js 20
Note over GitHub: Version bump 4.96.2 to 4.101.0
GitHub->>GitHub: Update package metadata
GitHub->>GitHub: Add new API types and methods
GitHub->>NPM: Publish updated package
GitHub->>JSR: Publish updated package
Users->>NPM: Install latest SDK
Users->>Users: Access new features (reinforcement fine-tuning, MCP, image generation)
Note over Users: Improved developer experience with added JSDoc examples
openai debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..a39d68281 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.102.0"
}
diff --git .stats.yml .stats.yml
index d92408173..41319e5e5 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 111
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-6af14840a810139bf407013167ce1c8fb21b6ef8eb0cc3db58b51af7d52c4b5a.yml
+openapi_spec_hash: 3241bde6b273cfec0035e522bd07985d
+config_hash: 7367b68a4e7db36885c1a886f57b17f6
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..b6254488a 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,88 @@
# Changelog
+## 4.102.0 (2025-05-21)
+
+Full Changelog: [v4.101.0...v4.102.0](https://github.com/openai/openai-node/compare/v4.101.0...v4.102.0)
+
+### Features
+
+* **api:** add container endpoint ([e973476](https://github.com/openai/openai-node/commit/e9734764625275c50ef612ff934804be8cb2adff))
+
+## 4.101.0 (2025-05-21)
+
+Full Changelog: [v4.100.0...v4.101.0](https://github.com/openai/openai-node/compare/v4.100.0...v4.101.0)
+
+### Features
+
+* **api:** new API tools ([fb4014f](https://github.com/openai/openai-node/commit/fb4014ffac7b220d37bd03c94fa745386b010bf0))
+
+
+### Chores
+
+* **docs:** grammar improvements ([7761cfb](https://github.com/openai/openai-node/commit/7761cfb0a8a56d056a73c046a6a613f66ada4abe))
+* **internal:** version bump ([b40e830](https://github.com/openai/openai-node/commit/b40e8302ec11683b6a360a050236ad1284afc760))
+
+## 4.100.0 (2025-05-16)
+
+Full Changelog: [v4.99.0...v4.100.0](https://github.com/openai/openai-node/compare/v4.99.0...v4.100.0)
+
+### Features
+
+* **api:** further updates for evals API ([3f6f248](https://github.com/openai/openai-node/commit/3f6f248191b45015924be76fd5154d149c4ed8a0))
+
+
+### Chores
+
+* **internal:** version bump ([5123fe0](https://github.com/openai/openai-node/commit/5123fe08a56f3d0040b1cc67129382f3eacc3cca))
+
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git SECURITY.md SECURITY.md
index 3b3bd8a66..4adb0c54f 100644
--- SECURITY.md
+++ SECURITY.md
@@ -16,13 +16,13 @@ before making any information public.
## Reporting Non-SDK Related Security Issues
If you encounter security issues that are not directly related to SDKs but pertain to the services
-or products provided by OpenAI please follow the respective company's security reporting guidelines.
+or products provided by OpenAI, please follow the respective company's security reporting guidelines.
### OpenAI Terms and Policies
Our Security Policy can be found at [Security Policy URL](https://openai.com/policies/coordinated-vulnerability-disclosure-policy).
-Please contact [email protected] for any questions or concerns regarding security of our services.
+Please contact [email protected] for any questions or concerns regarding the security of our services.
---
diff --git api.md api.md
index 49e6548a8..4e8187d0f 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -605,6 +645,10 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionWebSearch</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallGeneratingEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallPartialImageEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncludable</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncompleteEvent</a></code>
@@ -618,6 +662,14 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseInputMessageItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInputText</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDoneEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputAudio</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItemAddedEvent</a></code>
@@ -625,7 +677,13 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputMessage</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputRefusal</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputText</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseOutputTextAnnotationAddedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseQueuedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartAddedEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryTextDeltaEvent</a></code>
@@ -653,6 +711,7 @@ Methods:
- <code title="post /responses">client.responses.<a href="./src/resources/responses/responses.ts">create</a>({ ...params }) -> Response</code>
- <code title="get /responses/{response_id}">client.responses.<a href="./src/resources/responses/responses.ts">retrieve</a>(responseId, { ...params }) -> Response</code>
- <code title="delete /responses/{response_id}">client.responses.<a href="./src/resources/responses/responses.ts">del</a>(responseId) -> void</code>
+- <code title="post /responses/{response_id}/cancel">client.responses.<a href="./src/resources/responses/responses.ts">cancel</a>(responseId) -> void</code>
## InputItems
@@ -669,10 +728,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
@@ -719,3 +775,39 @@ Methods:
- <code title="get /evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}">client.evals.runs.outputItems.<a href="./src/resources/evals/runs/output-items.ts">retrieve</a>(evalId, runId, outputItemId) -> OutputItemRetrieveResponse</code>
- <code title="get /evals/{eval_id}/runs/{run_id}/output_items">client.evals.runs.outputItems.<a href="./src/resources/evals/runs/output-items.ts">list</a>(evalId, runId, { ...params }) -> OutputItemListResponsesPage</code>
+
+# Containers
+
+Types:
+
+- <code><a href="./src/resources/containers/containers.ts">ContainerCreateResponse</a></code>
+- <code><a href="./src/resources/containers/containers.ts">ContainerRetrieveResponse</a></code>
+- <code><a href="./src/resources/containers/containers.ts">ContainerListResponse</a></code>
+
+Methods:
+
+- <code title="post /containers">client.containers.<a href="./src/resources/containers/containers.ts">create</a>({ ...params }) -> ContainerCreateResponse</code>
+- <code title="get /containers/{container_id}">client.containers.<a href="./src/resources/containers/containers.ts">retrieve</a>(containerId) -> ContainerRetrieveResponse</code>
+- <code title="get /containers">client.containers.<a href="./src/resources/containers/containers.ts">list</a>({ ...params }) -> ContainerListResponsesPage</code>
+- <code title="delete /containers/{container_id}">client.containers.<a href="./src/resources/containers/containers.ts">del</a>(containerId) -> void</code>
+
+## Files
+
+Types:
+
+- <code><a href="./src/resources/containers/files/files.ts">FileCreateResponse</a></code>
+- <code><a href="./src/resources/containers/files/files.ts">FileRetrieveResponse</a></code>
+- <code><a href="./src/resources/containers/files/files.ts">FileListResponse</a></code>
+
+Methods:
+
+- <code title="post /containers/{container_id}/files">client.containers.files.<a href="./src/resources/containers/files/files.ts">create</a>(containerId, { ...params }) -> FileCreateResponse</code>
+- <code title="get /containers/{container_id}/files/{file_id}">client.containers.files.<a href="./src/resources/containers/files/files.ts">retrieve</a>(containerId, fileId) -> FileRetrieveResponse</code>
+- <code title="get /containers/{container_id}/files">client.containers.files.<a href="./src/resources/containers/files/files.ts">list</a>(containerId, { ...params }) -> FileListResponsesPage</code>
+- <code title="delete /containers/{container_id}/files/{file_id}">client.containers.files.<a href="./src/resources/containers/files/files.ts">del</a>(containerId, fileId) -> void</code>
+
+### Content
+
+Methods:
+
+- <code title="get /containers/{container_id}/files/{file_id}/content">client.containers.files.content.<a href="./src/resources/containers/files/content.ts">retrieve</a>(containerId, fileId) -> void</code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..a48e4e9ca 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.102.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..c1886271b 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.102.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..3b4b9a574 100644
--- src/index.ts
+++ src/index.ts
@@ -66,24 +66,31 @@ import {
import { Audio, AudioModel, AudioResponseFormat } from './resources/audio/audio';
import { Beta } from './resources/beta/beta';
import { Chat } from './resources/chat/chat';
+import {
+ ContainerCreateParams,
+ ContainerCreateResponse,
+ ContainerListParams,
+ ContainerListResponse,
+ ContainerListResponsesPage,
+ ContainerRetrieveResponse,
+ Containers,
+} from './resources/containers/containers';
import {
EvalCreateParams,
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,12 +312,14 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
uploads: API.Uploads = new API.Uploads(this);
responses: API.Responses = new API.Responses(this);
evals: API.Evals = new API.Evals(this);
+ containers: API.Containers = new API.Containers(this);
protected override defaultQuery(): Core.DefaultQuery | undefined {
return this._options.defaultQuery;
@@ -366,6 +375,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -376,6 +386,8 @@ OpenAI.Uploads = UploadsAPIUploads;
OpenAI.Responses = Responses;
OpenAI.Evals = Evals;
OpenAI.EvalListResponsesPage = EvalListResponsesPage;
+OpenAI.Containers = Containers;
+OpenAI.ContainerListResponsesPage = ContainerListResponsesPage;
export declare namespace OpenAI {
export type RequestOptions = Core.RequestOptions;
@@ -487,6 +499,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +545,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
@@ -546,6 +557,16 @@ export declare namespace OpenAI {
type EvalListParams as EvalListParams,
};
+ export {
+ Containers as Containers,
+ type ContainerCreateResponse as ContainerCreateResponse,
+ type ContainerRetrieveResponse as ContainerRetrieveResponse,
+ type ContainerListResponse as ContainerListResponse,
+ ContainerListResponsesPage as ContainerListResponsesPage,
+ type ContainerCreateParams as ContainerCreateParams,
+ type ContainerListParams as ContainerListParams,
+ };
+
export type AllModels = API.AllModels;
export type ChatModel = API.ChatModel;
export type ComparisonFilter = API.ComparisonFilter;
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/realtime.ts src/resources/beta/realtime/realtime.ts
index 5012b1edd..26fba883e 100644
--- src/resources/beta/realtime/realtime.ts
+++ src/resources/beta/realtime/realtime.ts
@@ -2571,7 +2571,7 @@ export interface TranscriptionSessionUpdatedEvent {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
session: TranscriptionSessionsAPI.TranscriptionSession;
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..83e8c47ad 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
@@ -29,7 +35,7 @@ export class TranscriptionSessions extends APIResource {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
export interface TranscriptionSession {
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git a/src/resources/containers.ts b/src/resources/containers.ts
new file mode 100644
index 000000000..c8f3a6456
--- /dev/null
+++ src/resources/containers.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './containers/index';
diff --git a/src/resources/containers/containers.ts b/src/resources/containers/containers.ts
new file mode 100644
index 000000000..66303dc61
--- /dev/null
+++ src/resources/containers/containers.ts
@@ -0,0 +1,291 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import { isRequestOptions } from '../../core';
+import * as Core from '../../core';
+import * as FilesAPI from './files/files';
+import {
+ FileCreateParams,
+ FileCreateResponse,
+ FileListParams,
+ FileListResponse,
+ FileListResponsesPage,
+ FileRetrieveResponse,
+ Files,
+} from './files/files';
+import { CursorPage, type CursorPageParams } from '../../pagination';
+
+export class Containers extends APIResource {
+ files: FilesAPI.Files = new FilesAPI.Files(this._client);
+
+ /**
+ * Create Container
+ */
+ create(
+ body: ContainerCreateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<ContainerCreateResponse> {
+ return this._client.post('/containers', { body, ...options });
+ }
+
+ /**
+ * Retrieve Container
+ */
+ retrieve(containerId: string, options?: Core.RequestOptions): Core.APIPromise<ContainerRetrieveResponse> {
+ return this._client.get(`/containers/${containerId}`, options);
+ }
+
+ /**
+ * List Containers
+ */
+ list(
+ query?: ContainerListParams,
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<ContainerListResponsesPage, ContainerListResponse>;
+ list(options?: Core.RequestOptions): Core.PagePromise<ContainerListResponsesPage, ContainerListResponse>;
+ list(
+ query: ContainerListParams | Core.RequestOptions = {},
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<ContainerListResponsesPage, ContainerListResponse> {
+ if (isRequestOptions(query)) {
+ return this.list({}, query);
+ }
+ return this._client.getAPIList('/containers', ContainerListResponsesPage, { query, ...options });
+ }
+
+ /**
+ * Delete Container
+ */
+ del(containerId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.delete(`/containers/${containerId}`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
+}
+
+export class ContainerListResponsesPage extends CursorPage<ContainerListResponse> {}
+
+export interface ContainerCreateResponse {
+ /**
+ * Unique identifier for the container.
+ */
+ id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the container was created.
+ */
+ created_at: number;
+
+ /**
+ * Name of the container.
+ */
+ name: string;
+
+ /**
+ * The type of this object.
+ */
+ object: string;
+
+ /**
+ * Status of the container (e.g., active, deleted).
+ */
+ status: string;
+
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ expires_after?: ContainerCreateResponse.ExpiresAfter;
+}
+
+export namespace ContainerCreateResponse {
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ export interface ExpiresAfter {
+ /**
+ * The reference point for the expiration.
+ */
+ anchor?: 'last_active_at';
+
+ /**
+ * The number of minutes after the anchor before the container expires.
+ */
+ minutes?: number;
+ }
+}
+
+export interface ContainerRetrieveResponse {
+ /**
+ * Unique identifier for the container.
+ */
+ id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the container was created.
+ */
+ created_at: number;
+
+ /**
+ * Name of the container.
+ */
+ name: string;
+
+ /**
+ * The type of this object.
+ */
+ object: string;
+
+ /**
+ * Status of the container (e.g., active, deleted).
+ */
+ status: string;
+
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ expires_after?: ContainerRetrieveResponse.ExpiresAfter;
+}
+
+export namespace ContainerRetrieveResponse {
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ export interface ExpiresAfter {
+ /**
+ * The reference point for the expiration.
+ */
+ anchor?: 'last_active_at';
+
+ /**
+ * The number of minutes after the anchor before the container expires.
+ */
+ minutes?: number;
+ }
+}
+
+export interface ContainerListResponse {
+ /**
+ * Unique identifier for the container.
+ */
+ id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the container was created.
+ */
+ created_at: number;
+
+ /**
+ * Name of the container.
+ */
+ name: string;
+
+ /**
+ * The type of this object.
+ */
+ object: string;
+
+ /**
+ * Status of the container (e.g., active, deleted).
+ */
+ status: string;
+
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ expires_after?: ContainerListResponse.ExpiresAfter;
+}
+
+export namespace ContainerListResponse {
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ export interface ExpiresAfter {
+ /**
+ * The reference point for the expiration.
+ */
+ anchor?: 'last_active_at';
+
+ /**
+ * The number of minutes after the anchor before the container expires.
+ */
+ minutes?: number;
+ }
+}
+
+export interface ContainerCreateParams {
+ /**
+ * Name of the container to create.
+ */
+ name: string;
+
+ /**
+ * Container expiration time in seconds relative to the 'anchor' time.
+ */
+ expires_after?: ContainerCreateParams.ExpiresAfter;
+
+ /**
+ * IDs of files to copy to the container.
+ */
+ file_ids?: Array<string>;
+}
+
+export namespace ContainerCreateParams {
+ /**
+ * Container expiration time in seconds relative to the 'anchor' time.
+ */
+ export interface ExpiresAfter {
+ /**
+ * Time anchor for the expiration time. Currently only 'last_active_at' is
+ * supported.
+ */
+ anchor: 'last_active_at';
+
+ minutes: number;
+ }
+}
+
+export interface ContainerListParams extends CursorPageParams {
+ /**
+ * Sort order by the `created_at` timestamp of the objects. `asc` for ascending
+ * order and `desc` for descending order.
+ */
+ order?: 'asc' | 'desc';
+}
+
+Containers.ContainerListResponsesPage = ContainerListResponsesPage;
+Containers.Files = Files;
+Containers.FileListResponsesPage = FileListResponsesPage;
+
+export declare namespace Containers {
+ export {
+ type ContainerCreateResponse as ContainerCreateResponse,
+ type ContainerRetrieveResponse as ContainerRetrieveResponse,
+ type ContainerListResponse as ContainerListResponse,
+ ContainerListResponsesPage as ContainerListResponsesPage,
+ type ContainerCreateParams as ContainerCreateParams,
+ type ContainerListParams as ContainerListParams,
+ };
+
+ export {
+ Files as Files,
+ type FileCreateResponse as FileCreateResponse,
+ type FileRetrieveResponse as FileRetrieveResponse,
+ type FileListResponse as FileListResponse,
+ FileListResponsesPage as FileListResponsesPage,
+ type FileCreateParams as FileCreateParams,
+ type FileListParams as FileListParams,
+ };
+}
diff --git a/src/resources/containers/files.ts b/src/resources/containers/files.ts
new file mode 100644
index 000000000..46a5299c1
--- /dev/null
+++ src/resources/containers/files.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './files/index';
diff --git a/src/resources/containers/files/content.ts b/src/resources/containers/files/content.ts
new file mode 100644
index 000000000..efb33a87f
--- /dev/null
+++ src/resources/containers/files/content.ts
@@ -0,0 +1,16 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+
+export class Content extends APIResource {
+ /**
+ * Retrieve Container File Content
+ */
+ retrieve(containerId: string, fileId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.get(`/containers/${containerId}/files/${fileId}/content`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
+}
diff --git a/src/resources/containers/files/files.ts b/src/resources/containers/files/files.ts
new file mode 100644
index 000000000..0c249df0f
--- /dev/null
+++ src/resources/containers/files/files.ts
@@ -0,0 +1,222 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import { isRequestOptions } from '../../../core';
+import * as Core from '../../../core';
+import * as ContentAPI from './content';
+import { Content } from './content';
+import { CursorPage, type CursorPageParams } from '../../../pagination';
+
+export class Files extends APIResource {
+ content: ContentAPI.Content = new ContentAPI.Content(this._client);
+
+ /**
+ * Create a Container File
+ *
+ * You can send either a multipart/form-data request with the raw file content, or
+ * a JSON request with a file ID.
+ */
+ create(
+ containerId: string,
+ body: FileCreateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<FileCreateResponse> {
+ return this._client.post(`/containers/${containerId}/files`, { body, ...options });
+ }
+
+ /**
+ * Retrieve Container File
+ */
+ retrieve(
+ containerId: string,
+ fileId: string,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<FileRetrieveResponse> {
+ return this._client.get(`/containers/${containerId}/files/${fileId}`, options);
+ }
+
+ /**
+ * List Container files
+ */
+ list(
+ containerId: string,
+ query?: FileListParams,
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<FileListResponsesPage, FileListResponse>;
+ list(
+ containerId: string,
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<FileListResponsesPage, FileListResponse>;
+ list(
+ containerId: string,
+ query: FileListParams | Core.RequestOptions = {},
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<FileListResponsesPage, FileListResponse> {
+ if (isRequestOptions(query)) {
+ return this.list(containerId, {}, query);
+ }
+ return this._client.getAPIList(`/containers/${containerId}/files`, FileListResponsesPage, {
+ query,
+ ...options,
+ });
+ }
+
+ /**
+ * Delete Container File
+ */
+ del(containerId: string, fileId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.delete(`/containers/${containerId}/files/${fileId}`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
+}
+
+export class FileListResponsesPage extends CursorPage<FileListResponse> {}
+
+export interface FileCreateResponse {
+ /**
+ * Unique identifier for the file.
+ */
+ id: string;
+
+ /**
+ * Size of the file in bytes.
+ */
+ bytes: number;
+
+ /**
+ * The container this file belongs to.
+ */
+ container_id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the file was created.
+ */
+ created_at: number;
+
+ /**
+ * The type of this object (`container.file`).
+ */
+ object: 'container.file';
+
+ /**
+ * Path of the file in the container.
+ */
+ path: string;
+
+ /**
+ * Source of the file (e.g., `user`, `assistant`).
+ */
+ source: string;
+}
+
+export interface FileRetrieveResponse {
+ /**
+ * Unique identifier for the file.
+ */
+ id: string;
+
+ /**
+ * Size of the file in bytes.
+ */
+ bytes: number;
+
+ /**
+ * The container this file belongs to.
+ */
+ container_id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the file was created.
+ */
+ created_at: number;
+
+ /**
+ * The type of this object (`container.file`).
+ */
+ object: 'container.file';
+
+ /**
+ * Path of the file in the container.
+ */
+ path: string;
+
+ /**
+ * Source of the file (e.g., `user`, `assistant`).
+ */
+ source: string;
+}
+
+export interface FileListResponse {
+ /**
+ * Unique identifier for the file.
+ */
+ id: string;
+
+ /**
+ * Size of the file in bytes.
+ */
+ bytes: number;
+
+ /**
+ * The container this file belongs to.
+ */
+ container_id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the file was created.
+ */
+ created_at: number;
+
+ /**
+ * The type of this object (`container.file`).
+ */
+ object: 'container.file';
+
+ /**
+ * Path of the file in the container.
+ */
+ path: string;
+
+ /**
+ * Source of the file (e.g., `user`, `assistant`).
+ */
+ source: string;
+}
+
+export interface FileCreateParams {
+ /**
+ * The File object (not file name) to be uploaded.
+ */
+ file?: Core.Uploadable;
+
+ /**
+ * Name of the file to create.
+ */
+ file_id?: string;
+}
+
+export interface FileListParams extends CursorPageParams {
+ /**
+ * Sort order by the `created_at` timestamp of the objects. `asc` for ascending
+ * order and `desc` for descending order.
+ */
+ order?: 'asc' | 'desc';
+}
+
+Files.FileListResponsesPage = FileListResponsesPage;
+Files.Content = Content;
+
+export declare namespace Files {
+ export {
+ type FileCreateResponse as FileCreateResponse,
+ type FileRetrieveResponse as FileRetrieveResponse,
+ type FileListResponse as FileListResponse,
+ FileListResponsesPage as FileListResponsesPage,
+ type FileCreateParams as FileCreateParams,
+ type FileListParams as FileListParams,
+ };
+
+ export { Content as Content };
+}
diff --git a/src/resources/containers/files/index.ts b/src/resources/containers/files/index.ts
new file mode 100644
index 000000000..7305ab706
--- /dev/null
+++ src/resources/containers/files/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Content } from './content';
+export {
+ FileListResponsesPage,
+ Files,
+ type FileCreateResponse,
+ type FileRetrieveResponse,
+ type FileListResponse,
+ type FileCreateParams,
+ type FileListParams,
+} from './files';
diff --git a/src/resources/containers/index.ts b/src/resources/containers/index.ts
new file mode 100644
index 000000000..91145bbe0
--- /dev/null
+++ src/resources/containers/index.ts
@@ -0,0 +1,20 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ ContainerListResponsesPage,
+ Containers,
+ type ContainerCreateResponse,
+ type ContainerRetrieveResponse,
+ type ContainerListResponse,
+ type ContainerCreateParams,
+ type ContainerListParams,
+} from './containers';
+export {
+ FileListResponsesPage,
+ Files,
+ type FileCreateResponse,
+ type FileRetrieveResponse,
+ type FileListResponse,
+ type FileCreateParams,
+ type FileListParams,
+} from './files/index';
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreat,eParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..05a656619 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -27,7 +28,8 @@ export class Evals extends APIResource {
/**
* Create the structure of an evaluation that can be used to test a model's
- * performance. An evaluation is a set of testing criteria and a datasource. After
+ * performance. An evaluation is a set of testing criteria and the config for a
+ * data source, which dictates the schema of the data used in the evaluation. After
* creating an evaluation, you can run it on different models and model parameters.
* We support several types of graders and datasources. For more information, see
* the [Evals guide](https://platform.openai.com/docs/guides/evals).
@@ -104,88 +106,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -210,90 +131,13 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalCreateResponse {
/**
@@ -309,7 +153,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +182,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +244,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -465,7 +258,7 @@ export namespace EvalCreateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalRetrieveResponse {
/**
@@ -481,7 +274,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +303,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +365,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -637,7 +379,7 @@ export namespace EvalRetrieveResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalUpdateResponse {
/**
@@ -653,7 +395,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +424,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +486,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -809,7 +500,7 @@ export namespace EvalUpdateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalListResponse {
/**
@@ -825,7 +516,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +545,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +607,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -985,17 +625,21 @@ export interface EvalDeleteResponse {
export interface EvalCreateParams {
/**
- * The configuration for the data source used for the evaluation runs.
+ * The configuration for the data source used for the evaluation runs. Dictates the
+ * schema of the data used in the evaluation.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
- * A list of graders for all eval runs in this group.
+ * A list of graders for all eval runs in this group. Graders can reference
+ * variables in the data source using double curly braces notation, like
+ * `{{item.variable_name}}`. To reference the model's output, use the `sample`
+ * namespace (ie, `{{sample.output_text}}`).
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +688,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +703,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored_completions`.
+ */
+ type: 'stored_completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1067,7 +725,7 @@ export namespace EvalCreateParams {
export interface LabelModel {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
input: Array<LabelModel.SimpleInputMessage | LabelModel.EvalItem>;
@@ -1154,29 +812,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +834,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +880,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..e761e2160 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -19,7 +19,9 @@ export class Runs extends APIResource {
outputItems: OutputItemsAPI.OutputItems = new OutputItemsAPI.OutputItems(this._client);
/**
- * Create a new evaluation run. This is the endpoint that will kick off grading.
+ * Kicks off a new run for a given evaluation, specifying the data source, and what
+ * model configuration to use to test. The datasource will be validated against the
+ * schema specified in the config of the evaluation.
*/
create(
evalId: string,
@@ -85,7 +87,7 @@ export class RunListResponsesPage extends CursorPage<RunListResponse> {}
*/
export interface CreateEvalCompletionsRunDataSource {
/**
- * A StoredCompletionsRunDataSource configuration describing a set of filters
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalCompletionsRunDataSource.FileContent
@@ -97,6 +99,12 @@ export interface CreateEvalCompletionsRunDataSource {
*/
type: 'completions';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalCompletionsRunDataSource.Template
| CreateEvalCompletionsRunDataSource.ItemReference;
@@ -185,7 +193,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<ResponsesAPI.EasyInputMessage | Template.Message>;
@@ -241,7 +249,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"
*/
item_reference: string;
@@ -279,6 +287,9 @@ export namespace CreateEvalCompletionsRunDataSource {
* eval
*/
export interface CreateEvalJSONLRunDataSource {
+ /**
+ * Determines what populates the `item` namespace in the data source.
+ */
source: CreateEvalJSONLRunDataSource.FileContent | CreateEvalJSONLRunDataSource.FileID;
/**
@@ -356,7 +367,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +434,34 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +503,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -505,14 +516,8 @@ export namespace RunCreateResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +544,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -553,7 +563,7 @@ export namespace RunCreateResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -621,7 +631,7 @@ export namespace RunCreateResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -749,7 +759,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +826,34 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +895,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -898,14 +908,8 @@ export namespace RunRetrieveResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +936,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -946,7 +955,7 @@ export namespace RunRetrieveResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1014,7 +1023,7 @@ export namespace RunRetrieveResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1139,10 +1148,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1215,34 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1284,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1291,14 +1297,8 @@ export namespace RunListResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1325,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1339,7 +1344,7 @@ export namespace RunListResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1407,7 +1412,7 @@ export namespace RunListResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1543,7 +1548,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1615,34 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1684,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1692,14 +1697,8 @@ export namespace RunCancelResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1725,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1740,7 +1744,7 @@ export namespace RunCancelResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1808,7 +1812,7 @@ export namespace RunCancelResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1947,7 +1951,7 @@ export namespace RunCreateParams {
*/
export interface CreateEvalResponsesRunDataSource {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalResponsesRunDataSource.FileContent
@@ -1955,10 +1959,16 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalResponsesRunDataSource.Template
| CreateEvalResponsesRunDataSource.ItemReference;
@@ -2013,12 +2023,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2032,14 +2036,8 @@ export namespace RunCreateParams {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2064,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -2080,7 +2083,7 @@ export namespace RunCreateParams {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -2148,7 +2151,7 @@ export namespace RunCreateParams {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparamet,ers,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.mu,ltipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..d1f816a84 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -22,6 +22,15 @@ export {
type CompletionCreateParamsNonStreaming,
type CompletionCreateParamsStreaming,
} from './completions';
+export {
+ ContainerListResponsesPage,
+ Containers,
+ type ContainerCreateResponse,
+ type ContainerRetrieveResponse,
+ type ContainerListResponse,
+ type ContainerCreateParams,
+ type ContainerListParams,
+} from './containers/containers';
export {
Embeddings,
type CreateEmbeddingResponse,
@@ -33,10 +42,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +63,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..40dd9a6fb 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -37,7 +37,13 @@ export type ParsedResponseOutputItem<ParsedT> =
| ResponseFileSearchToolCall
| ResponseFunctionWebSearch
| ResponseComputerToolCall
- | ResponseReasoningItem;
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
export interface ParsedResponse<ParsedT> extends Response {
output: Array<ParsedResponseOutputItem<ParsedT>>;
@@ -61,6 +67,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +104,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +131,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -130,12 +158,33 @@ export class Responses extends APIResource {
/**
* Creates a model response stream
*/
+
stream<Params extends ResponseStreamParams, ParsedT = ExtractParsedContentFromParams<Params>>(
body: Params,
options?: Core.RequestOptions,
): ResponseStream<ParsedT> {
return ResponseStream.createResponse<ParsedT>(this._client, body, options);
}
+
+ /**
+ * Cancels a model response with the given ID. Only responses created with the
+ * `background` parameter set to `true` can be cancelled.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ *
+ * @example
+ * ```ts
+ * await client.responses.cancel(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
+ */
+
+ cancel(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.post(`/responses/${responseId}/cancel`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
}
export class ResponseItemsPage extends CursorPage<ResponseItem> {}
@@ -158,7 +207,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +258,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +307,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -392,6 +441,12 @@ export interface Response {
*/
top_p: number | null;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* An upper bound for the number of tokens that can be generated for a response,
* including visible output tokens and
@@ -437,7 +492,7 @@ export interface Response {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
status?: ResponseStatus;
@@ -496,6 +551,11 @@ export interface ResponseAudioDeltaEvent {
*/
delta: string;
+ /**
+ * A sequence number for this chunk of the stream response.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.delta`.
*/
@@ -506,6 +566,11 @@ export interface ResponseAudioDeltaEvent {
* Emitted when the audio response is complete.
*/
export interface ResponseAudioDoneEvent {
+ /**
+ * The sequence number of the delta.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.done`.
*/
@@ -521,6 +586,11 @@ export interface ResponseAudioTranscriptDeltaEvent {
*/
delta: string;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.transcript.delta`.
*/
@@ -531,6 +601,11 @@ export interface ResponseAudioTranscriptDeltaEvent {
* Emitted when the full audio transcript is completed.
*/
export interface ResponseAudioTranscriptDoneEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.transcript.done`.
*/
@@ -551,6 +626,11 @@ export interface ResponseCodeInterpreterCallCodeDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.code.delta`.
*/
@@ -571,6 +651,11 @@ export interface ResponseCodeInterpreterCallCodeDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.code.done`.
*/
@@ -591,6 +676,11 @@ export interface ResponseCodeInterpreterCallCompletedEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.completed`.
*/
@@ -611,6 +701,11 @@ export interface ResponseCodeInterpreterCallInProgressEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.in_progress`.
*/
@@ -631,6 +726,11 @@ export interface ResponseCodeInterpreterCallInterpretingEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.interpreting`.
*/
@@ -665,6 +765,11 @@ export interface ResponseCodeInterpreterToolCall {
* The type of the code interpreter tool call. Always `code_interpreter_call`.
*/
type: 'code_interpreter_call';
+
+ /**
+ ,* The ID of the container used to run the code.
+ */
+ container_id?: string;
}
export namespace ResponseCodeInterpreterToolCall {
@@ -719,6 +824,11 @@ export interface ResponseCompletedEvent {
*/
response: Response;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.completed`.
*/
@@ -1101,6 +1211,11 @@ export interface ResponseContentPartAddedEvent {
*/
part: ResponseOutputText | ResponseOutputRefusal;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.content_part.added`.
*/
@@ -1131,6 +1246,11 @@ export interface ResponseContentPartDoneEvent {
*/
part: ResponseOutputText | ResponseOutputRefusal;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.content_part.done`.
*/
@@ -1146,6 +1266,11 @@ export interface ResponseCreatedEvent {
*/
response: Response;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.created`.
*/
@@ -1204,6 +1329,11 @@ export interface ResponseErrorEvent {
*/
param: string | null;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `error`.
*/
@@ -1219,6 +1349,11 @@ export interface ResponseFailedEvent {
*/
response: Response;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.failed`.
*/
@@ -1239,6 +1374,11 @@ export interface ResponseFileSearchCallCompletedEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.file_search_call.completed`.
*/
@@ -1259,6 +1399,11 @@ export interface ResponseFileSearchCallInProgressEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.file_search_call.in_progress`.
*/
@@ -1279,6 +1424,11 @@ export interface ResponseFileSearchCallSearchingEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.file_search_call.searching`.
*/
@@ -1429,6 +1579,11 @@ export interface ResponseFunctionCallArgumentsDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.function_call_arguments.delta`.
*/
@@ -1454,6 +1609,11 @@ export interface ResponseFunctionCallArgumentsDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
type: 'response.function_call_arguments.done';
}
@@ -1557,6 +1717,119 @@ export interface ResponseFunctionWebSearch {
type: 'web_search_call';
}
+/**
+ * Emitted when an image generation tool call has completed and the final image is
+ * available.
+ */
+export interface ResponseImageGenCallCompletedEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.completed'.
+ */
+ type: 'response.image_generation_call.completed';
+}
+
+/**
+ * Emitted when an image generation tool call is actively generating an image
+ * (intermediate state).
+ */
+export interface ResponseImageGenCallGeneratingEvent {
+ /**
+ * The unique identifier of the image generation item b,eing processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.generating'.
+ */
+ type: 'response.image_generation_call.generating';
+}
+
+/**
+ * Emitted when an image generation tool call is in progress.
+ */
+export interface ResponseImageGenCallInProgressEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.in_progress'.
+ */
+ type: 'response.image_generation_call.in_progress';
+}
+
+/**
+ * Emitted when a partial image is available during image generation streaming.
+ */
+export interface ResponseImageGenCallPartialImageEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * Base64-encoded partial image data, suitable for rendering as an image.
+ */
+ partial_image_b64: string;
+
+ /**
+ * 0-based index for the partial image (backend is 1-based, but this is 0-based for
+ * the user).
+ */
+ partial_image_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.partial_image'.
+ */
+ type: 'response.image_generation_call.partial_image';
+}
+
/**
* Emitted when the response is in progress.
*/
@@ -1566,6 +1839,11 @@ export interface ResponseInProgressEvent {
*/
response: Response;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.in_progress`.
*/
@@ -1581,11 +1859,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1596,6 +1880,11 @@ export interface ResponseIncompleteEvent {
*/
response: Response;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.incomplete`.
*/
@@ -1650,7 +1939,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1956,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1704,6 +1993,14 @@ export type ResponseInputItem =
| ResponseFunctio,nToolCall
| ResponseInputItem.FunctionCallOutput
| ResponseReasoningItem
+ | ResponseInputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseInputItem.LocalShellCall
+ | ResponseInputItem.LocalShellCallOutput
+ | ResponseInputItem.McpListTools
+ | ResponseInputItem.McpApprovalRequest
+ | ResponseInputItem.McpApprovalResponse
+ | ResponseInputItem.McpCall
| ResponseInputItem.ItemReference;
export namespace ResponseInputItem {
@@ -1758,19 +2055,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +2083,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,104 +2115,824 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
- * An internal identifier for an item to reference.
+ * An image generation request made by the model.
*/
- export interface ItemReference {
+ export interface ImageGenerationCall {
/**
- * The ID of the item to reference.
+ * The unique ID of the image generation call.
*/
id: string;
/**
- * The type of item to reference. Always `item_reference`.
+ * The generated image encoded in base64.
*/
- type: 'item_reference';
- }
-}
+ result: string | null;
-/**
- * A list of one or many input items to the model, containing different content
- * types.
- */
-export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
-export interface ResponseInputMessageItem {
- /**
- * The unique ID of the message input.
- */
- id: string;
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
/**
- * A list of one or many input items to the model, containing different content
- * types.
+ * A tool call to run a command on the local shell.
*/
- content: ResponseInputMessageContentList;
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
- /**
- * The role of the message input. One of `user`, `system`, or `developer`.
- */
- role: 'user' | 'system' | 'developer';
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
- /**
- * The status of item. One of `in_progress`, `completed`, or `incomplete`.
- * Populated when items are returned via API.
- */
- status?: 'in_progress' | 'completed' | 'incomplete';
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
- /**
- * The type of the message input. Always set to `message`.
- */
- type?: 'message';
-}
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
-/**
- * A text input to the model.
- */
-export interface ResponseInputText {
- /**
- * The text input to the model.
- */
- text: string;
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
- /**
- * The type of the input item. Always `input_text`.
- */
- type: 'input_text';
-}
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
-/**
- * Content item used to generate a response.
- */
-export type ResponseItem =
- | ResponseInputMessageItem
- | ResponseOutputMessage
- | ResponseFileSearchToolCall
- | ResponseComputerToolCall
- | ResponseComputerToolCallOutputItem
- | ResponseFunctionWebSearch
- | ResponseFunctionToolCallItem
- | ResponseFunctionToolCallOutputItem;
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
-/**
- * An audio output from the model.
- */
-export interface ResponseOutputAudio {
- /**
- * Base64-encoded audio data from the model.
- */
- data: string;
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
- /**
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * The output of a local shell tool call.
+ */
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
+
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
+
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * The unique ID of the approval response
+ */
+ id?: string | null;
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * An internal identifier for an item to reference.
+ */
+ export interface ItemReference {
+ /**
+ * The ID of the item to reference.
+ */
+ id: string;
+
+ /**
+ * The type of item to reference. Always `item_reference`.
+ */
+ type?: 'item_reference' | null;
+ }
+}
+
+/**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+
+export interface ResponseInputMessageItem {
+ /**
+ * The unique ID of the message input.
+ */
+ id: string;
+
+ /**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+ content: ResponseInputMessageContentList;
+
+ /**
+ * The role of the message input. One of `user`, `system`, or `developer`.
+ */
+ role: 'user' | 'system' | 'developer';
+
+ /**
+ * The status of item. One of `in_progress`, `completed`, or `incomplete`.
+ * Populated when items are returned via API.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the message input. Always set to `message`.
+ */
+ type?: 'message';
+}
+
+/**
+ * A text input to the model.
+ */
+export interface ResponseInputText {
+ /**
+ * The text input to the model.
+ */
+ text: string;
+
+ /**
+ * The type of the input item. Always `input_text`.
+ */
+ type: 'input_text';
+}
+
+/**
+ * Content item used to generate a response.
+ */
+export type ResponseItem =
+ | ResponseInputMessageItem
+ | ResponseOutputMessage
+ | ResponseFileSearchToolCall
+ | ResponseComputerToolCall
+ | ResponseComputerToolCallOutputItem
+ | ResponseFunctionWebSearch
+ | ResponseFunctionToolCallItem
+ | ResponseFunctionToolCallOutputItem
+ | ResponseItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseItem.LocalShellCall
+ | ResponseItem.LocalShellCallOutput
+ | ResponseItem.McpListTools
+ | ResponseItem.McpApprovalRequest
+ | ResponseItem.McpApprovalResponse
+ | ResponseItem.McpCall;
+
+export namespace ResponseItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * The output of a local shell tool call.
+ */
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
+
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
+
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The unique ID of the approval response
+ */
+ id: string;
+
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the arguments of an MCP tool
+ * call.
+ */
+export interface ResponseMcpCallArgumentsDeltaEvent {
+ /**
+ * The partial update to the arguments for the MCP tool call.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_delta'.
+ */
+ type: 'response.mcp_call.arguments_delta';
+}
+
+/**
+ * Emitted when the arguments for an MCP tool call are finalized.
+ */
+export interface ResponseMcpCallArgumentsDoneEvent {
+ /**
+ * The finalized arguments for the MCP tool call.
+ */
+ arguments: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_done'.
+ */
+ type: 'response.mcp_call.arguments_done';
+}
+
+/**
+ * Emitted when an MCP tool call has completed successfully.
+ */
+export interface ResponseMcpCallCompletedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.completed'.
+ */
+ type: 'response.mcp_call.completed';
+}
+
+/**
+ * Emitted when an MCP tool call has failed.
+ */
+export interface ResponseMcpCallFailedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.failed'.
+ */
+ type: 'response.mcp_call.failed';
+}
+
+/**
+ * Emitted when an MCP tool call is in progress.
+ */
+export interface ResponseMcpCallInProgressEvent {
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.in_progress'.
+ */
+ type: 'response.mcp_call.in_progress';
+}
+
+/**
+ * Emitted when the list of available MCP tools has been successfully retrieved.
+ */
+export interface ResponseMcpListToolsCompletedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.completed'.
+ */
+ type: 'response.mcp_list_tools.completed';
+}
+
+/**
+ * Emitted when the attempt to list available MCP tools has failed.
+ */
+export interface ResponseMcpListToolsFailedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.failed'.
+ */
+ type: 'response.mcp_list_tools.failed';
+}
+
+/**
+ * Emitted when the system is in the process of retrieving the list of available
+ * MCP tools.
+ */
+export interface ResponseMcpListToolsInProgressEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.in_progress'.
+ */
+ type: 'response.mcp_list_tools.in_progress';
+}
+
+/**
+ * An audio output from the model.
+ */
+export interface ResponseOutputAudio {
+ /**
+ * Base64-encoded audio data from the model.
+ */
+ data: string;
+
+ /**
* The transcript of the audio data from the model.
*/
transcript: string;
@@ -1935,7 +2952,234 @@ export type ResponseOutputItem =
| ResponseFunctionToolCall
| ResponseFunctionWebSearch
| ResponseComputerToolCall
- | ResponseReasoningItem;
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
+
+export namespace ResponseOutputItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+}
/**
* Emitted when a new output item is added.
@@ -1951,6 +3195,11 @@ export interface ResponseOutputItemAddedEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_item.added`.
*/
@@ -1971,6 +3220,11 @@ export interface ResponseOutputItemDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_item.done`.
*/
@@ -2075,51 +3329,183 @@ export namespace ResponseOutputText {
*/
end_index: number;
- /**
- * The index of the first character of the URL citation in the message.
- */
- start_index: number;
+ /**
+ * The index of the first character of the URL citation in the message.
+ */
+ start_index: number;
+
+ /**
+ * The title of the web resource.
+ */
+ title: string;
+
+ /**
+ * The type of the URL citation. Always `url_citation`.
+ */
+ type: 'url_citation';
+
+ /**
+ * The URL of the web resource.
+ */
+ url: string;
+ }
+
+ /**
+ * A path to a file.
+ */
+ export interface FilePath {
+ /**
+ * The ID of the file.
+ */
+ file_id: string;
+
+ /**
+ * The index of the file in the list of files.
+ */
+ index: number;
+
+ /**
+ * The type of the file path. Always `file_path`.
+ */
+ type: 'file_path';
+ }
+}
+
+/**
+ * Emitted when an annotation is added to output text content.
+ */
+export interface ResponseOutputTextAnnotationAddedEvent {
+ /**
+ * The annotation object being added. (See annotation schema for details.)
+ */
+ annotation: unknown;
+
+ /**
+ * The index of the annotation within the content part.
+ */
+ annotation_index: number;
+
+ /**
+ * The index of the content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The unique identifier of the item to which the annotation is being added.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.output_text_annotation.added'.
+ */
+ type: 'response.output_text_annotation.added';
+}
+
+/**
+ * Emitted when a response is queued and waiting to be processed.
+ */
+export interface ResponseQueuedEvent {
+ /**
+ * The full response object that is queued.
+ */
+ response: Response;
+
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.queued'.
+ */
+ type: 'response.queued';
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the reasoning content.
+ */
+export interface ResponseReasoningDeltaEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The partial update to the reasoning content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which reasoning is being updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning.delta'.
+ */
+ type: 'response.reasoning.delta';
+}
- /**
- * The title of the web resource.
- */
- title: string;
+/**
+ * Emitted when the reasoning content is finalized for an item.
+ */
+export interface ResponseReasoningDoneEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
- /**
- * The type of the URL citation. Always `url_citation`.
- */
- type: 'url_citation';
+ /**
+ * The unique identifier of the item for which reasoning is finalized.
+ */
+ item_id: string;
- /**
- * The URL of the web resource.
- */
- url: string;
- }
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
/**
- * A path to a file.
+ * The sequence number of this event.
*/
- export interface FilePath {
- /**
- * The ID of the file.
- */
- file_id: string;
+ sequence_number: number;
- /**
- * The index of the file in the list of files.
- */
- index: number;
+ /**
+ * The finalized reasoning text.
+ */
+ text: string;
- /**
- * The type of the file path. Always `file_path`.
- */
- type: 'file_path';
- }
+ /**
+ * The type of the event. Always 'response.reasoning.done'.
+ */
+ type: 'response.reasoning.done';
}
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +3523,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2158,6 +3550,77 @@ export namespace ResponseReasoningItem {
}
}
+/**
+ * Emitted when there is a delta (partial update) to the reasoning summary content.
+ */
+export interface ResponseReasoningSummaryDeltaEvent {
+ /**
+ * The partial update to the reasoning summary content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which the reasoning summary is being
+ * updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.delta'.
+ */
+ type: 'response.reasoning_summary.delta';
+}
+
+/**
+ * Emitted when the reasoning summary content is finalized for an item.
+ */
+export interface ResponseReasoningSummaryDoneEvent {
+ /**
+ * The unique identifier of the item for which the reasoning summary is finalized.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The finalized reasoning summary text.
+ */
+ text: string;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.done'.
+ */
+ type: 'response.reasoning_summary.done';
+}
+
/**
* Emitted when a new reasoning summary part is added.
*/
@@ -2177,6 +3640,11 @@ export interface ResponseReasoningSummaryPartAddedEvent {
*/
part: ResponseReasoningSummaryPartAddedEvent.Part;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2224,6 +3692,11 @@ export interface ResponseReasoningSummaryPartDoneEvent {
*/
part: ResponseReasoningSummaryPartDoneEvent.Part;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2271,6 +3744,11 @@ export interface ResponseReasoningSummaryTextDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2296,6 +3774,11 @@ export interface ResponseReasoningSummaryTextDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2336,6 +3819,11 @@ export interface ResponseRefusalDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.refusal.delta`.
*/
@@ -2366,6 +3854,11 @@ export interface ResponseRefusalDoneEvent {
*/
refusal: string;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.refusal.done`.
*/
@@ -2374,9 +3867,9 @@ export interface ResponseRefusalDoneEvent {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
-export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'incomplete';
+export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'cancelled' | 'queued' | 'incomplete';
/**
* Emitted when there is a partial audio response.
@@ -2417,7 +3910,25 @@ export type ResponseStreamEvent =
| ResponseTextDoneEvent
| ResponseWebSearchCallCompletedEvent
| ResponseWebSearchCallInProgressEvent
- | ResponseWebSearchCallSearchingEvent;
+ | ResponseWebSearchCallSearchingEvent
+ | ResponseImageGenCallCompletedEvent
+ | ResponseImageGenCallGeneratingEvent
+ | ResponseImageGenCallInProgressEvent
+ | ResponseImageGenCallPartialImageEvent
+ | ResponseMcpCallArgumentsDeltaEvent
+ | ResponseMcpCallArgumentsDoneEvent
+ | ResponseMcpCallCompletedEvent
+ | ResponseMcpCallFailedEvent
+ | ResponseMcpCallInProgressEvent
+ | ResponseMcpListToolsCompletedEvent
+ | ResponseMcpListToolsFailedEvent
+ | ResponseMcpListToolsInProgressEvent
+ | ResponseOutputTextAnnotationAddedEvent
+ | ResponseQueuedEvent
+ | ResponseReasoningDeltaEvent
+ | ResponseReasoningDoneEvent
+ | ResponseReasoningSummaryDeltaEvent
+ | ResponseReasoningSummaryDoneEvent;
/**
* Emitted when a text annotation is added.
@@ -2451,6 +3962,11 @@ export interface ResponseTextAnnotationDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_text.annotation.added`.
*/
@@ -2579,6 +4095,11 @@ export interface ResponseTextDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_text.delta`.
*/
@@ -2604,6 +4125,11 @@ export interface ResponseTextDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The text content that is finalized.
*/
@@ -2730,11 +4256,231 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool =
+ | FunctionTool
+ | FileSearchTool
+ | WebSearchTool
+ | ComputerTool
+ | Tool.Mcp
+ | Tool.CodeInterpreter
+ | Tool.ImageGeneration
+ | Tool.LocalShell;
+
+export namespace Tool {
+ /**
+ * Give the model access to additional tools via remote Model Context Protocol
+ * (MCP) servers.
+ * [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp).
+ */
+ export interface Mcp {
+ /**
+ * A label for this MCP server, used to identify it in tool calls.
+ */
+ server_label: string;
+
+ /**
+ * The URL for the MCP server.
+ */
+ server_url: string;
+
+ /**
+ * The type of the MCP tool. Always `mcp`.
+ */
+ type: 'mcp';
+
+ /**
+ * List of allowed tool names or a filter object.
+ */
+ allowed_tools?: Array<string> | Mcp.McpAllowedToolsFilter | null;
+
+ /**
+ * Optional HTTP headers to send to the MCP server. Use for authentication or other
+ * purposes.
+ */
+ headers?: Record<string, string> | null;
+
+ /**
+ * Specify which of the MCP server's tools require approval.
+ */
+ require_approval?: Mcp.McpToolApprovalFilter | 'always' | 'never' | null;
+ }
+
+ export namespace Mcp {
+ /**
+ * A filter object to specify which tools are allowed.
+ */
+ export interface McpAllowedToolsFilter {
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export interface McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ always?: McpToolApprovalFilter.Always;
+
+ /**
+ * A list of tools that never require approval.
+ */
+ never?: McpToolApprovalFilter.Never;
+
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export namespace McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ export interface Always {
+ /**
+ * List of tools that require approval.
+ */
+ tool_names?: Array<string>;
+ }
+
+ /**
+ * A list of tools that never require approval.
+ */
+ export interface Never {
+ /**
+ * List of tools that do not require approval.
+ */
+ tool_names?: Array<string>;
+ }
+ }
+ }
+
+ /**
+ * A tool that runs Python code to help generate a response to a prompt.
+ */
+ export interface CodeInterpreter {
+ /**
+ * The code interpreter container. Can be a container ID or an object that
+ * specifies uploaded file IDs to make available to your code.
+ */
+ container: string | CodeInterpreter.CodeInterpreterToolAuto;
+
+ /**
+ * The type of the code interpreter tool. Always `code_interpreter`.
+ */
+ type: 'code_interpreter';
+ }
+
+ export namespace CodeInterpreter {
+ /**
+ * Configuration for a code interpreter container. Optionally specify the IDs of
+ * the files to run the code on.
+ */
+ export interface CodeInterpreterToolAuto {
+ /**
+ * Always `auto`.
+ */
+ type: 'auto';
+
+ /**
+ * An optional list of uploaded files to make available to your code.
+ */
+ file_ids?: Array<string>;
+ }
+ }
+
+ /**
+ * A tool that generates images using a model like `gpt-image-1`.
+ */
+ export interface ImageGeneration {
+ /**
+ * The type of the image generation tool. Always `image_generation`.
+ */
+ type: 'image_generation';
+
+ /**
+ * Background type for the generated image. One of `transparent`, `opaque`, or
+ * `auto`. Default: `auto`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto';
+
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ input_image_mask?: ImageGeneration.InputImageMask;
+
+ /**
+ * The image generation model to use. Default: `gpt-image-1`.
+ */
+ model?: 'gpt-image-1';
+
+ /**
+ * Moderation level for the generated image. Default: `auto`.
+ */
+ moderation?: 'auto' | 'low';
+
+ /**
+ * Compression level for the output image. Default: 100.
+ */
+ output_compression?: number;
+
+ /**
+ * The output format of the generated image. One of `png`, `webp`, or `jpeg`.
+ * Default: `png`.
+ */
+ output_format?: 'png' | 'webp' | 'jpeg';
+
+ /**
+ * Number of partial images to generate in streaming mode, from 0 (default value)
+ * to 3.
+ */
+ partial_images?: number;
+
+ /**
+ * The quality of the generated image. One of `low`, `medium`, `high`, or `auto`.
+ * Default: `auto`.
+ */
+ quality?: 'low' | 'medium' | 'high' | 'auto';
+
+ /**
+ * The size of the generated image. One of `1024x1024`, `1024x1536`, `1536x1024`,
+ * or `auto`. Default: `auto`.
+ */
+ size?: '1024x1024' | '1024x1536' | '1536x1024' | 'auto';
+ }
+
+ export namespace ImageGeneration {
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ export interface InputImageMask {
+ /**
+ * File ID for the mask image.
+ */
+ file_id?: string;
+
+ /**
+ * Base64-encoded mask image.
+ */
+ image_url?: string;
+ }
+ }
+
+ /**
+ * A tool that allows the model to execute shell commands in a local environment.
+ */
+ export interface LocalShell {
+ /**
+ * The type of the local shell tool. Always `local_shell`.
+ */
+ type: 'local_shell';
+ }
+}
/**
* Use this option to force the model to call a specific function.
@@ -2777,8 +4523,18 @@ export interface ToolChoiceTypes {
* - `file_search`
* - `web_search_preview`
* - `computer_use_preview`
- */
- type: 'file_search' | 'web_search_preview' | 'computer_use_preview' | 'web_search_preview_2025_03_11';
+ * - `code_interpreter`
+ * - `mcp`
+ * - `image_generation`
+ */
+ type:
+ | 'file_search'
+ | 'web_search_preview'
+ | 'computer_use_preview'
+ | 'web_search_preview_2025_03_11'
+ | 'image_generation'
+ | 'code_interpreter'
+ | 'mcp';
}
/**
@@ -2788,10 +4544,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +4555,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +4574,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2860,6 +4620,12 @@ export interface ResponseCreateParamsBase {
*/
model: Shared.ResponsesModel;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* Specify additional output data to include in the model response. Currently
* supported values are:
@@ -2869,6 +4635,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
@@ -3104,6 +4875,10 @@ export declare namespace Responses {
type ResponseFunctionToolCallItem as ResponseFunctionToolCallItem,
type ResponseFunctionToolCallOutputItem as ResponseFunctionToolCallOutputItem,
type ResponseFunctionWebSearch as ResponseFunctionWebSearch,
+ type ResponseImageGenCallCompletedEvent as ResponseImageGenCallCompletedEvent,
+ type ResponseImageGenCallGeneratingEvent as ResponseImageGenCallGeneratingEvent,
+ type ResponseImageGenCallInProgressEvent as ResponseImageGenCallInProgressEvent,
+ type ResponseImageGenCallPartialImageEvent as ResponseImageGenCallPartialImageEvent,
type ResponseInProgressEvent as ResponseInProgressEvent,
type ResponseIncludable as ResponseIncludable,
type ResponseIncompleteEvent as ResponseIncompleteEvent,
@@ -3117,6 +4892,14 @@ export declare namespace Responses {
type ResponseInputMessageItem as ResponseInputMessageItem,
type ResponseInputText as ResponseInputText,
type ResponseItem as ResponseItem,
+ type ResponseMcpCallArgumentsDeltaEvent as ResponseMcpCallArgumentsDeltaEvent,
+ type ResponseMcpCallArgumentsDoneEvent as ResponseMcpCallArgumentsDoneEvent,
+ type ResponseMcpCallCompletedEvent as ResponseMcpCallCompletedEvent,
+ type ResponseMcpCallFailedEvent as ResponseMcpCallFailedEvent,
+ type ResponseMcpCallInProgressEvent as ResponseMcpCallInProgressEvent,
+ type ResponseMcpListToolsCompletedEvent as ResponseMcpListToolsCompletedEvent,
+ type ResponseMcpListToolsFailedEvent as ResponseMcpListToolsFailedEvent,
+ type ResponseMcpListToolsInProgressEvent as ResponseMcpListToolsInProgressEvent,
type ResponseOutputAudio as ResponseOutputAudio,
type ResponseOutputItem as ResponseOutputItem,
type ResponseOutputItemAddedEvent as ResponseOutputItemAddedEvent,
@@ -3124,7 +4907,13 @@ export declare namespace Responses {
type ResponseOutputMessage as ResponseOutputMessage,
type ResponseOutputRefusal as ResponseOutputRefusal,
type ResponseOutputText as ResponseOutputText,
+ type ResponseOutputTextAnnotationAddedEvent as ResponseOutputTextAnnotationAddedEvent,
+ type ResponseQueuedEvent as ResponseQueuedEvent,
+ type ResponseReasoningDeltaEvent as ResponseReasoningDeltaEvent,
+ type ResponseReasoningDoneEvent as ResponseReasoningDoneEvent,
type ResponseReasoningItem as ResponseReasoningItem,
+ type ResponseReasoningSummaryDeltaEvent as ResponseReasoningSummaryDeltaEvent,
+ type ResponseReasoningSummaryDoneEvent as ResponseReasoningSummaryDoneEvent,
type ResponseReasoningSummaryPartAddedEvent as ResponseReasoningSummaryPartAddedEvent,
type ResponseReasoningSummaryPartDoneEvent as ResponseReasoningSummaryPartDoneEvent,
type ResponseReasoningSummaryTextDeltaEvent as ResponseReasoningSummaryTextDeltaEvent,
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..180cea6da 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.102.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/containers/containers.test.ts b/tests/api-resources/containers/containers.test.ts
new file mode 100644
index 000000000..11c6f166c
--- /dev/null
+++ tests/api-resources/containers/containers.test.ts
@@ -0,0 +1,94 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource containers', () => {
+ test('create: only required params', async () => {
+ const responsePromise = client.containers.create({ name: 'name' });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('create: required and optional params', async () => {
+ const response = await client.containers.create({
+ name: 'name',
+ expires_after: { anchor: 'last_active_at', minutes: 0 },
+ file_ids: ['string'],
+ });
+ });
+
+ test('retrieve', async () => {
+ const responsePromise = client.containers.retrieve('container_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.retrieve('container_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('list', async () => {
+ const responsePromise = client.containers.list();
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('list: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(client.containers.list({ path: '/_stainless_unknown_path' })).rejects.toThrow(
+ OpenAI.NotFoundError,
+ );
+ });
+
+ test('list: request options and params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.list(
+ { after: 'after', limit: 0, order: 'asc' },
+ { path: '/_stainless_unknown_path' },
+ ),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('del', async () => {
+ const responsePromise = client.containers.del('container_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('del: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(client.containers.del('container_id', { path: '/_stainless_unknown_path' })).rejects.toThrow(
+ OpenAI.NotFoundError,
+ );
+ });
+});
diff --git a/tests/api-resources/containers/files/content.test.ts b/tests/api-resources/containers/files/content.test.ts
new file mode 100644
index 000000000..9215c001a
--- /dev/null
+++ tests/api-resources/containers/files/content.test.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource content', () => {
+ test('retrieve', async () => {
+ const responsePromise = client.containers.files.content.retrieve('container_id', 'file_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.content.retrieve('container_id', 'file_id', {
+ path: '/_stainless_unknown_path',
+ }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+});
diff --git a/tests/api-resources/containers/files/files.test.ts b/tests/api-resources/containers/files/files.test.ts
new file mode 100644
index 000000000..79f47e9ad
--- /dev/null
+++ tests/api-resources/containers/files/files.test.ts
@@ -0,0 +1,87 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource files', () => {
+ test('create', async () => {
+ const responsePromise = client.containers.files.create('container_id', {});
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve', async () => {
+ const responsePromise = client.containers.files.retrieve('container_id', 'file_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.retrieve('container_id', 'file_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('list', async () => {
+ const responsePromise = client.containers.files.list('container_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('list: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.list('container_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('list: request options and params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.list(
+ 'container_id',
+ { after: 'after', limit: 0, order: 'asc' },
+ { path: '/_stainless_unknown_path' },
+ ),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('del', async () => {
+ const responsePromise = client.containers.files.del('container_id', 'file_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('del: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.del('container_id', 'file_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+});
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndR,esponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
diff --git tests/api-resources/responses/responses.test.ts tests/api-resources/responses/responses.test.ts
index cf7e9cf3c..f8e653c52 100644
--- tests/api-resources/responses/responses.test.ts
+++ tests/api-resources/responses/responses.test.ts
@@ -24,6 +24,7 @@ describe('resource responses', () => {
const response = await client.responses.create({
input: 'string',
model: 'gpt-4o',
+ background: true,
include: ['file_search_call.results'],
instructions: 'instructions',
max_output_tokens: 0,
@@ -39,11 +40,11 @@ describe('resource responses', () => {
tool_choice: 'none',
tools: [
{
- type: 'file_search',
- vector_store_ids: ['string'],
- filters: { key: 'key', type: 'eq', value: 'string' },
- max_num_results: 0,
- ranking_options: { ranker: 'auto', score_threshold: 0 },
+ name: 'name',
+ parameters: { foo: 'bar' },
+ strict: true,
+ type: 'function',
+ description: 'description',
},
],
top_p: 1,
@@ -100,4 +101,22 @@ describe('resource responses', () => {
client.responses.del('resp_677efb5139a88190b512bc3fef8e535d', { path: '/_stainless_unknown_path' }),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('cancel', async () => {
+ const responsePromise = client.responses.cancel('resp_677efb5139a88190b512bc3fef8e535d');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('cancel: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.responses.cancel('resp_677efb5139a88190b512bc3fef8e535d', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
Description
This pull request includes several updates and additions to the OpenAI Node.js library, primarily focusing on upgrading dependencies, adding new features, improving documentation, and fixing minor issues.
Possible Issues
None identified.
Security Hotspots
None identified.
Changes
Changes
Workflow and Configuration Changes
-
.github/workflows/ci.yml
- Updated Node.js version for CI to
20
- Updated Node.js version for CI to
-
.github/workflows/publish-jsr.yml
- Updated Node.js version for publishing to
20
- Updated Node.js version for publishing to
-
.github/workflows/publish-npm.yml
- Updated Node.js version for publishing to
20
- Updated Node.js version for publishing to
-
.release-please-manifest.json
- Version bump to
4.102.0
- Version bump to
-
.stats.yml
- Updated configured endpoints and OpenAPI spec URL/hash
-
CHANGELOG.md
- Added entries for multiple new versions
Documentation Changes
-
README.md, SECURITY.md, api.md, realtime.md
- Grammar corrections and enhancements
- New and updated examples and descriptions
API Changes
- Added:
- src/resources/containers/containers.ts: Container management (create, retrieve, list, delete)
- src/resources/containers/files/files.ts: Container file management (create, retrieve, list, delete)
- src/resources/fine-tuning/alpha/graders.ts: Graders for fine-tuning jobs
- src/resources/graders/grader-models.ts: Grader model types
- src/resources/evals/evals.ts: Updated evals with new graders and data source configurations
- Various minor additions and improvements related to responses for handling events and output formats
- Updated:
-
package.json, jsr.json, src/version.ts: Version bump to
4.102.0 - src/resources/evals/evals.ts: Added new testing criteria and data source configurations
- src/resources/chat/completions/completions.ts, src/resources/responses/responses.ts: Added new methods and options for API resources
- tests/api-resources: Added new tests for updated and new API resources
-
package.json, jsr.json, src/version.ts: Version bump to
sequenceDiagram
participant Developer
participant GithubCI as Github CI
participant Node18 as Node.js 18
participant Node20 as Node.js 20
participant Backend as OpenAI Backend
Developer->>GithubCI: Create/update pull request
GithubCI->>Node18: Remove node v18
GithubCI--XNode20: Add node v20 for CI and publishing
Developer->>GithubCI: Merge pull request
GithubCI->>Backend: Run tests and publish updates
Developer-->>Backend: Utilize new endpoints and methods
anthropic debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..a39d68281 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.102.0"
}
diff --git .stats.yml .stats.yml
index d92408173..41319e5e5 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 111
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-6af14840a810139bf407013167ce1c8fb21b6ef8eb0cc3db58b51af7d52c4b5a.yml
+openapi_spec_hash: 3241bde6b273cfec0035e522bd07985d
+config_hash: 7367b68a4e7db36885c1a886f57b17f6
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..b6254488a 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,88 @@
# Changelog
+## 4.102.0 (2025-05-21)
+
+Full Changelog: [v4.101.0...v4.102.0](https://github.com/openai/openai-node/compare/v4.101.0...v4.102.0)
+
+### Features
+
+* **api:** add container endpoint ([e973476](https://github.com/openai/openai-node/commit/e9734764625275c50ef612ff934804be8cb2adff))
+
+## 4.101.0 (2025-05-21)
+
+Full Changelog: [v4.100.0...v4.101.0](https://github.com/openai/openai-node/compare/v4.100.0...v4.101.0)
+
+### Features
+
+* **api:** new API tools ([fb4014f](https://github.com/openai/openai-node/commit/fb4014ffac7b220d37bd03c94fa745386b010bf0))
+
+
+### Chores
+
+* **docs:** grammar improvements ([7761cfb](https://github.com/openai/openai-node/commit/7761cfb0a8a56d056a73c046a6a613f66ada4abe))
+* **internal:** version bump ([b40e830](https://github.com/openai/openai-node/commit/b40e8302ec11683b6a360a050236ad1284afc760))
+
+## 4.100.0 (2025-05-16)
+
+Full Changelog: [v4.99.0...v4.100.0](https://github.com/openai/openai-node/compare/v4.99.0...v4.100.0)
+
+### Features
+
+* **api:** further updates for evals API ([3f6f248](https://github.com/openai/openai-node/commit/3f6f248191b45015924be76fd5154d149c4ed8a0))
+
+
+### Chores
+
+* **internal:** version bump ([5123fe0](https://github.com/openai/openai-node/commit/5123fe08a56f3d0040b1cc67129382f3eacc3cca))
+
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git SECURITY.md SECURITY.md
index 3b3bd8a66..4adb0c54f 100644
--- SECURITY.md
+++ SECURITY.md
@@ -16,13 +16,13 @@ before making any information public.
## Reporting Non-SDK Related Security Issues
If you encounter security issues that are not directly related to SDKs but pertain to the services
-or products provided by OpenAI please follow the respective company's security reporting guidelines.
+or products provided by OpenAI, please follow the respective company's security reporting guidelines.
### OpenAI Terms and Policies
Our Security Policy can be found at [Security Policy URL](https://openai.com/policies/coordinated-vulnerability-disclosure-policy).
-Please contact [email protected] for any questions or concerns regarding security of our services.
+Please contact [email protected] for any questions or concerns regarding the security of our services.
---
diff --git api.md api.md
index 49e6548a8..4e8187d0f 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -605,6 +645,10 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionWebSearch</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallGeneratingEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallPartialImageEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncludable</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncompleteEvent</a></code>
@@ -618,6 +662,14 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseInputMessageItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInputText</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDoneEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputAudio</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItemAddedEvent</a></code>
@@ -625,7 +677,13 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputMessage</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputRefusal</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputText</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseOutputTextAnnotationAddedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseQueuedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartAddedEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryTextDeltaEvent</a></code>
@@ -653,6 +711,7 @@ Methods:
- <code title="post /responses">client.responses.<a href="./src/resources/responses/responses.ts">create</a>({ ...params }) -> Response</code>
- <code title="get /responses/{response_id}">client.responses.<a href="./src/resources/responses/responses.ts">retrieve</a>(responseId, { ...params }) -> Response</code>
- <code title="delete /responses/{response_id}">client.responses.<a href="./src/resources/responses/responses.ts">del</a>(responseId) -> void</code>
+- <code title="post /responses/{response_id}/cancel">client.responses.<a href="./src/resources/responses/responses.ts">cancel</a>(responseId) -> void</code>
## InputItems
@@ -669,10 +728,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
@@ -719,3 +775,39 @@ Methods:
- <code title="get /evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}">client.evals.runs.outputItems.<a href="./src/resources/evals/runs/output-items.ts">retrieve</a>(evalId, runId, outputItemId) -> OutputItemRetrieveResponse</code>
- <code title="get /evals/{eval_id}/runs/{run_id}/output_items">client.evals.runs.outputItems.<a href="./src/resources/evals/runs/output-items.ts">list</a>(evalId, runId, { ...params }) -> OutputItemListResponsesPage</code>
+
+# Containers
+
+Types:
+
+- <code><a href="./src/resources/containers/containers.ts">ContainerCreateResponse</a></code>
+- <code><a href="./src/resources/containers/containers.ts">ContainerRetrieveResponse</a></code>
+- <code><a href="./src/resources/containers/containers.ts">ContainerListResponse</a></code>
+
+Methods:
+
+- <code title="post /containers">client.containers.<a href="./src/resources/containers/containers.ts">create</a>({ ...params }) -> ContainerCreateResponse</code>
+- <code title="get /containers/{container_id}">client.containers.<a href="./src/resources/containers/containers.ts">retrieve</a>(containerId) -> ContainerRetrieveResponse</code>
+- <code title="get /containers">client.containers.<a href="./src/resources/containers/containers.ts">list</a>({ ...params }) -> ContainerListResponsesPage</code>
+- <code title="delete /containers/{container_id}">client.containers.<a href="./src/resources/containers/containers.ts">del</a>(containerId) -> void</code>
+
+## Files
+
+Types:
+
+- <code><a href="./src/resources/containers/files/files.ts">FileCreateResponse</a></code>
+- <code><a href="./src/resources/containers/files/files.ts">FileRetrieveResponse</a></code>
+- <code><a href="./src/resources/containers/files/files.ts">FileListResponse</a></code>
+
+Methods:
+
+- <code title="post /containers/{container_id}/files">client.containers.files.<a href="./src/resources/containers/files/files.ts">create</a>(containerId, { ...params }) -> FileCreateResponse</code>
+- <code title="get /containers/{container_id}/files/{file_id}">client.containers.files.<a href="./src/resources/containers/files/files.ts">retrieve</a>(containerId, fileId) -> FileRetrieveResponse</code>
+- <code title="get /containers/{container_id}/files">client.containers.files.<a href="./src/resources/containers/files/files.ts">list</a>(containerId, { ...params }) -> FileListResponsesPage</code>
+- <code title="delete /containers/{container_id}/files/{file_id}">client.containers.files.<a href="./src/resources/containers/files/files.ts">del</a>(containerId, fileId) -> void</code>
+
+### Content
+
+Methods:
+
+- <code title="get /containers/{container_id}/files/{file_id}/content">client.containers.files.content.<a href="./src/resources/containers/files/content.ts">retrieve</a>(containerId, fileId) -> void</code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..a48e4e9ca 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.102.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..c1886271b 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.102.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..3b4b9a574 100644
--- src/index.ts
+++ src/index.ts
@@ -66,24 +66,31 @@ import {
import { Audio, AudioModel, AudioResponseFormat } from './resources/audio/audio';
import { Beta } from './resources/beta/beta';
import { Chat } from './resources/chat/chat';
+import {
+ ContainerCreateParams,
+ ContainerCreateResponse,
+ ContainerListParams,
+ ContainerListResponse,
+ ContainerListResponsesPage,
+ ContainerRetrieveResponse,
+ Containers,
+} from './resources/containers/containers';
import {
EvalCreateParams,
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,12 +312,14 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
uploads: API.Uploads = new API.Uploads(this);
responses: API.Responses = new API.Responses(this);
evals: API.Evals = new API.Evals(this);
+ containers: API.Containers = new API.Containers(this);
protected override defaultQuery(): Core.DefaultQuery | undefined {
return this._options.defaultQuery;
@@ -366,6 +375,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -376,6 +386,8 @@ OpenAI.Uploads = UploadsAPIUploads;
OpenAI.Responses = Responses;
OpenAI.Evals = Evals;
OpenAI.EvalListResponsesPage = EvalListResponsesPage;
+OpenAI.Containers = Containers;
+OpenAI.ContainerListResponsesPage = ContainerListResponsesPage;
export declare namespace OpenAI {
export type RequestOptions = Core.RequestOptions;
@@ -487,6 +499,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +545,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
@@ -546,6 +557,16 @@ export declare namespace OpenAI {
type EvalListParams as EvalListParams,
};
+ export {
+ Containers as Containers,
+ type ContainerCreateResponse as ContainerCreateResponse,
+ type ContainerRetrieveResponse as ContainerRetrieveResponse,
+ type ContainerListResponse as ContainerListResponse,
+ ContainerListResponsesPage as ContainerListResponsesPage,
+ type ContainerCreateParams as ContainerCreateParams,
+ type ContainerListParams as ContainerListParams,
+ };
+
export type AllModels = API.AllModels;
export type ChatModel = API.ChatModel;
export type ComparisonFilter = API.ComparisonFilter;
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/realtime.ts src/resources/beta/realtime/realtime.ts
index 5012b1edd..26fba883e 100644
--- src/resources/beta/realtime/realtime.ts
+++ src/resources/beta/realtime/realtime.ts
@@ -2571,7 +2571,7 @@ export interface TranscriptionSessionUpdatedEvent {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
session: TranscriptionSessionsAPI.TranscriptionSession;
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..83e8c47ad 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
@@ -29,7 +35,7 @@ export class TranscriptionSessions extends APIResource {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
export interface TranscriptionSession {
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git a/src/resources/containers.ts b/src/resources/containers.ts
new file mode 100644
index 000000000..c8f3a6456
--- /dev/null
+++ src/resources/containers.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './containers/index';
diff --git a/src/resources/containers/containers.ts b/src/resources/containers/containers.ts
new file mode 100644
index 000000000..66303dc61
--- /dev/null
+++ src/resources/containers/containers.ts
@@ -0,0 +1,291 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import { isRequestOptions } from '../../core';
+import * as Core from '../../core';
+import * as FilesAPI from './files/files';
+import {
+ FileCreateParams,
+ FileCreateResponse,
+ FileListParams,
+ FileListResponse,
+ FileListResponsesPage,
+ FileRetrieveResponse,
+ Files,
+} from './files/files';
+import { CursorPage, type CursorPageParams } from '../../pagination';
+
+export class Containers extends APIResource {
+ files: FilesAPI.Files = new FilesAPI.Files(this._client);
+
+ /**
+ * Create Container
+ */
+ create(
+ body: ContainerCreateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<ContainerCreateResponse> {
+ return this._client.post('/containers', { body, ...options });
+ }
+
+ /**
+ * Retrieve Container
+ */
+ retrieve(containerId: string, options?: Core.RequestOptions): Core.APIPromise<ContainerRetrieveResponse> {
+ return this._client.get(`/containers/${containerId}`, options);
+ }
+
+ /**
+ * List Containers
+ */
+ list(
+ query?: ContainerListParams,
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<ContainerListResponsesPage, ContainerListResponse>;
+ list(options?: Core.RequestOptions): Core.PagePromise<ContainerListResponsesPage, ContainerListResponse>;
+ list(
+ query: ContainerListParams | Core.RequestOptions = {},
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<ContainerListResponsesPage, ContainerListResponse> {
+ if (isRequestOptions(query)) {
+ return this.list({}, query);
+ }
+ return this._client.getAPIList('/containers', ContainerListResponsesPage, { query, ...options });
+ }
+
+ /**
+ * Delete Container
+ */
+ del(containerId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.delete(`/containers/${containerId}`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
+}
+
+export class ContainerListResponsesPage extends CursorPage<ContainerListResponse> {}
+
+export interface ContainerCreateResponse {
+ /**
+ * Unique identifier for the container.
+ */
+ id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the container was created.
+ */
+ created_at: number;
+
+ /**
+ * Name of the container.
+ */
+ name: string;
+
+ /**
+ * The type of this object.
+ */
+ object: string;
+
+ /**
+ * Status of the container (e.g., active, deleted).
+ */
+ status: string;
+
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ expires_after?: ContainerCreateResponse.ExpiresAfter;
+}
+
+export namespace ContainerCreateResponse {
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ export interface ExpiresAfter {
+ /**
+ * The reference point for the expiration.
+ */
+ anchor?: 'last_active_at';
+
+ /**
+ * The number of minutes after the anchor before the container expires.
+ */
+ minutes?: number;
+ }
+}
+
+export interface ContainerRetrieveResponse {
+ /**
+ * Unique identifier for the container.
+ */
+ id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the container was created.
+ */
+ created_at: number;
+
+ /**
+ * Name of the container.
+ */
+ name: string;
+
+ /**
+ * The type of this object.
+ */
+ object: string;
+
+ /**
+ * Status of the container (e.g., active, deleted).
+ */
+ status: string;
+
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ expires_after?: ContainerRetrieveResponse.ExpiresAfter;
+}
+
+export namespace ContainerRetrieveResponse {
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ export interface ExpiresAfter {
+ /**
+ * The reference point for the expiration.
+ */
+ anchor?: 'last_active_at';
+
+ /**
+ * The number of minutes after the anchor before the container expires.
+ */
+ minutes?: number;
+ }
+}
+
+export interface ContainerListResponse {
+ /**
+ * Unique identifier for the container.
+ */
+ id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the container was created.
+ */
+ created_at: number;
+
+ /**
+ * Name of the container.
+ */
+ name: string;
+
+ /**
+ * The type of this object.
+ */
+ object: string;
+
+ /**
+ * Status of the container (e.g., active, deleted).
+ */
+ status: string;
+
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ expires_after?: ContainerListResponse.ExpiresAfter;
+}
+
+export namespace ContainerListResponse {
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ export interface ExpiresAfter {
+ /**
+ * The reference point for the expiration.
+ */
+ anchor?: 'last_active_at';
+
+ /**
+ * The number of minutes after the anchor before the container expires.
+ */
+ minutes?: number;
+ }
+}
+
+export interface ContainerCreateParams {
+ /**
+ * Name of the container to create.
+ */
+ name: string;
+
+ /**
+ * Container expiration time in seconds relative to the 'anchor' time.
+ */
+ expires_after?: ContainerCreateParams.ExpiresAfter;
+
+ /**
+ * IDs of files to copy to the container.
+ */
+ file_ids?: Array<string>;
+}
+
+export namespace ContainerCreateParams {
+ /**
+ * Container expiration time in seconds relative to the 'anchor' time.
+ */
+ export interface ExpiresAfter {
+ /**
+ * Time anchor for the expiration time. Currently only 'last_active_at' is
+ * supported.
+ */
+ anchor: 'last_active_at';
+
+ minutes: number;
+ }
+}
+
+export interface ContainerListParams extends CursorPageParams {
+ /**
+ * Sort order by the `created_at` timestamp of the objects. `asc` for ascending
+ * order and `desc` for descending order.
+ */
+ order?: 'asc' | 'desc';
+}
+
+Containers.ContainerListResponsesPage = ContainerListResponsesPage;
+Containers.Files = Files;
+Containers.FileListResponsesPage = FileListResponsesPage;
+
+export declare namespace Containers {
+ export {
+ type ContainerCreateResponse as ContainerCreateResponse,
+ type ContainerRetrieveResponse as ContainerRetrieveResponse,
+ type ContainerListResponse as ContainerListResponse,
+ ContainerListResponsesPage as ContainerListResponsesPage,
+ type ContainerCreateParams as ContainerCreateParams,
+ type ContainerListParams as ContainerListParams,
+ };
+
+ export {
+ Files as Files,
+ type FileCreateResponse as FileCreateResponse,
+ type FileRetrieveResponse as FileRetrieveResponse,
+ type FileListResponse as FileListResponse,
+ FileListResponsesPage as FileListResponsesPage,
+ type FileCreateParams as FileCreateParams,
+ type FileListParams as FileListParams,
+ };
+}
diff --git a/src/resources/containers/files.ts b/src/resources/containers/files.ts
new file mode 100644
index 000000000..46a5299c1
--- /dev/null
+++ src/resources/containers/files.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './files/index';
diff --git a/src/resources/containers/files/content.ts b/src/resources/containers/files/content.ts
new file mode 100644
index 000000000..efb33a87f
--- /dev/null
+++ src/resources/containers/files/content.ts
@@ -0,0 +1,16 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+
+export class Content extends APIResource {
+ /**
+ * Retrieve Container File Content
+ */
+ retrieve(containerId: string, fileId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.get(`/containers/${containerId}/files/${fileId}/content`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
+}
diff --git a/src/resources/containers/files/files.ts b/src/resources/containers/files/files.ts
new file mode 100644
index 000000000..0c249df0f
--- /dev/null
+++ src/resources/containers/files/files.ts
@@ -0,0 +1,222 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import { isRequestOptions } from '../../../core';
+import * as Core from '../../../core';
+import * as ContentAPI from './content';
+import { Content } from './content';
+import { CursorPage, type CursorPageParams } from '../../../pagination';
+
+export class Files extends APIResource {
+ content: ContentAPI.Content = new ContentAPI.Content(this._client);
+
+ /**
+ * Create a Container File
+ *
+ * You can send either a multipart/form-data request with the raw file content, or
+ * a JSON request with a file ID.
+ */
+ create(
+ containerId: string,
+ body: FileCreateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<FileCreateResponse> {
+ return this._client.post(`/containers/${containerId}/files`, { body, ...options });
+ }
+
+ /**
+ * Retrieve Container File
+ */
+ retrieve(
+ containerId: string,
+ fileId: string,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<FileRetrieveResponse> {
+ return this._client.get(`/containers/${containerId}/files/${fileId}`, options);
+ }
+
+ /**
+ * List Container files
+ */
+ list(
+ containerId: string,
+ query?: FileListParams,
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<FileListResponsesPage, FileListResponse>;
+ list(
+ containerId: string,
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<FileListResponsesPage, FileListResponse>;
+ list(
+ containerId: string,
+ query: FileListParams | Core.RequestOptions = {},
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<FileListResponsesPage, FileListResponse> {
+ if (isRequestOptions(query)) {
+ return this.list(containerId, {}, query);
+ }
+ return this._client.getAPIList(`/containers/${containerId}/files`, FileListResponsesPage, {
+ query,
+ ...options,
+ });
+ }
+
+ /**
+ * Delete Container File
+ */
+ del(containerId: string, fileId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.delete(`/containers/${containerId}/files/${fileId}`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
+}
+
+export class FileListResponsesPage extends CursorPage<FileListResponse> {}
+
+export interface FileCreateResponse {
+ /**
+ * Unique identifier for the file.
+ */
+ id: string;
+
+ /**
+ * Size of the file in bytes.
+ */
+ bytes: number;
+
+ /**
+ * The container this file belongs to.
+ */
+ container_id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the file was created.
+ */
+ created_at: number;
+
+ /**
+ * The type of this object (`container.file`).
+ */
+ object: 'container.file';
+
+ /**
+ * Path of the file in the container.
+ */
+ path: string;
+
+ /**
+ * Source of the file (e.g., `user`, `assistant`).
+ */
+ source: string;
+}
+
+export interface FileRetrieveResponse {
+ /**
+ * Unique identifier for the file.
+ */
+ id: string;
+
+ /**
+ * Size of the file in bytes.
+ */
+ bytes: number;
+
+ /**
+ * The container this file belongs to.
+ */
+ container_id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the file was created.
+ */
+ created_at: number;
+
+ /**
+ * The type of this object (`container.file`).
+ */
+ object: 'container.file';
+
+ /**
+ * Path of the file in the container.
+ */
+ path: string;
+
+ /**
+ * Source of the file (e.g., `user`, `assistant`).
+ */
+ source: string;
+}
+
+export interface FileListResponse {
+ /**
+ * Unique identifier for the file.
+ */
+ id: string;
+
+ /**
+ * Size of the file in bytes.
+ */
+ bytes: number;
+
+ /**
+ * The container this file belongs to.
+ */
+ container_id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the file was created.
+ */
+ created_at: number;
+
+ /**
+ * The type of this object (`container.file`).
+ */
+ object: 'container.file';
+
+ /**
+ * Path of the file in the container.
+ */
+ path: string;
+
+ /**
+ * Source of the file (e.g., `user`, `assistant`).
+ */
+ source: string;
+}
+
+export interface FileCreateParams {
+ /**
+ * The File object (not file name) to be uploaded.
+ */
+ file?: Core.Uploadable;
+
+ /**
+ * Name of the file to create.
+ */
+ file_id?: string;
+}
+
+export interface FileListParams extends CursorPageParams {
+ /**
+ * Sort order by the `created_at` timestamp of the objects. `asc` for ascending
+ * order and `desc` for descending order.
+ */
+ order?: 'asc' | 'desc';
+}
+
+Files.FileListResponsesPage = FileListResponsesPage;
+Files.Content = Content;
+
+export declare namespace Files {
+ export {
+ type FileCreateResponse as FileCreateResponse,
+ type FileRetrieveResponse as FileRetrieveResponse,
+ type FileListResponse as FileListResponse,
+ FileListResponsesPage as FileListResponsesPage,
+ type FileCreateParams as FileCreateParams,
+ type FileListParams as FileListParams,
+ };
+
+ export { Content as Content };
+}
diff --git a/src/resources/containers/files/index.ts b/src/resources/containers/files/index.ts
new file mode 100644
index 000000000..7305ab706
--- /dev/null
+++ src/resources/containers/files/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Content } from './content';
+export {
+ FileListResponsesPage,
+ Files,
+ type FileCreateResponse,
+ type FileRetrieveResponse,
+ type FileListResponse,
+ type FileCreateParams,
+ type FileListParams,
+} from './files';
diff --git a/src/resources/containers/index.ts b/src/resources/containers/index.ts
new file mode 100644
index 000000000..91145bbe0
--- /dev/null
+++ src/resources/containers/index.ts
@@ -0,0 +1,20 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ ContainerListResponsesPage,
+ Containers,
+ type ContainerCreateResponse,
+ type ContainerRetrieveResponse,
+ type ContainerListResponse,
+ type ContainerCreateParams,
+ type ContainerListParams,
+} from './containers';
+export {
+ FileListResponsesPage,
+ Files,
+ type FileCreateResponse,
+ type FileRetrieveResponse,
+ type FileListResponse,
+ type FileCreateParams,
+ type FileListParams,
+} from './files/index';
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreat,eParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..05a656619 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -27,7 +28,8 @@ export class Evals extends APIResource {
/**
* Create the structure of an evaluation that can be used to test a model's
- * performance. An evaluation is a set of testing criteria and a datasource. After
+ * performance. An evaluation is a set of testing criteria and the config for a
+ * data source, which dictates the schema of the data used in the evaluation. After
* creating an evaluation, you can run it on different models and model parameters.
* We support several types of graders and datasources. For more information, see
* the [Evals guide](https://platform.openai.com/docs/guides/evals).
@@ -104,88 +106,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -210,90 +131,13 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalCreateResponse {
/**
@@ -309,7 +153,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +182,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +244,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -465,7 +258,7 @@ export namespace EvalCreateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalRetrieveResponse {
/**
@@ -481,7 +274,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +303,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +365,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -637,7 +379,7 @@ export namespace EvalRetrieveResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalUpdateResponse {
/**
@@ -653,7 +395,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +424,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +486,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -809,7 +500,7 @@ export namespace EvalUpdateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalListResponse {
/**
@@ -825,7 +516,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +545,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +607,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -985,17 +625,21 @@ export interface EvalDeleteResponse {
export interface EvalCreateParams {
/**
- * The configuration for the data source used for the evaluation runs.
+ * The configuration for the data source used for the evaluation runs. Dictates the
+ * schema of the data used in the evaluation.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
- * A list of graders for all eval runs in this group.
+ * A list of graders for all eval runs in this group. Graders can reference
+ * variables in the data source using double curly braces notation, like
+ * `{{item.variable_name}}`. To reference the model's output, use the `sample`
+ * namespace (ie, `{{sample.output_text}}`).
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +688,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +703,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored_completions`.
+ */
+ type: 'stored_completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1067,7 +725,7 @@ export namespace EvalCreateParams {
export interface LabelModel {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
input: Array<LabelModel.SimpleInputMessage | LabelModel.EvalItem>;
@@ -1154,29 +812,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +834,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +880,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..e761e2160 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -19,7 +19,9 @@ export class Runs extends APIResource {
outputItems: OutputItemsAPI.OutputItems = new OutputItemsAPI.OutputItems(this._client);
/**
- * Create a new evaluation run. This is the endpoint that will kick off grading.
+ * Kicks off a new run for a given evaluation, specifying the data source, and what
+ * model configuration to use to test. The datasource will be validated against the
+ * schema specified in the config of the evaluation.
*/
create(
evalId: string,
@@ -85,7 +87,7 @@ export class RunListResponsesPage extends CursorPage<RunListResponse> {}
*/
export interface CreateEvalCompletionsRunDataSource {
/**
- * A StoredCompletionsRunDataSource configuration describing a set of filters
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalCompletionsRunDataSource.FileContent
@@ -97,6 +99,12 @@ export interface CreateEvalCompletionsRunDataSource {
*/
type: 'completions';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalCompletionsRunDataSource.Template
| CreateEvalCompletionsRunDataSource.ItemReference;
@@ -185,7 +193,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<ResponsesAPI.EasyInputMessage | Template.Message>;
@@ -241,7 +249,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"
*/
item_reference: string;
@@ -279,6 +287,9 @@ export namespace CreateEvalCompletionsRunDataSource {
* eval
*/
export interface CreateEvalJSONLRunDataSource {
+ /**
+ * Determines what populates the `item` namespace in the data source.
+ */
source: CreateEvalJSONLRunDataSource.FileContent | CreateEvalJSONLRunDataSource.FileID;
/**
@@ -356,7 +367,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +434,34 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +503,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -505,14 +516,8 @@ export namespace RunCreateResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +544,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -553,7 +563,7 @@ export namespace RunCreateResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -621,7 +631,7 @@ export namespace RunCreateResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -749,7 +759,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +826,34 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +895,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -898,14 +908,8 @@ export namespace RunRetrieveResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +936,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -946,7 +955,7 @@ export namespace RunRetrieveResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1014,7 +1023,7 @@ export namespace RunRetrieveResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1139,10 +1148,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1215,34 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1284,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1291,14 +1297,8 @@ export namespace RunListResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1325,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1339,7 +1344,7 @@ export namespace RunListResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1407,7 +1412,7 @@ export namespace RunListResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1543,7 +1548,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1615,34 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1684,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1692,14 +1697,8 @@ export namespace RunCancelResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1725,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1740,7 +1744,7 @@ export namespace RunCancelResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1808,7 +1812,7 @@ export namespace RunCancelResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1947,7 +1951,7 @@ export namespace RunCreateParams {
*/
export interface CreateEvalResponsesRunDataSource {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalResponsesRunDataSource.FileContent
@@ -1955,10 +1959,16 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalResponsesRunDataSource.Template
| CreateEvalResponsesRunDataSource.ItemReference;
@@ -2013,12 +2023,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2032,14 +2036,8 @@ export namespace RunCreateParams {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2064,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -2080,7 +2083,7 @@ export namespace RunCreateParams {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -2148,7 +2151,7 @@ export namespace RunCreateParams {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparamet,ers,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..d1f816a84 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -22,6 +22,15 @@ export {
type CompletionCreateParamsNonStreaming,
type CompletionCreateParamsStreaming,
} from './completions';
+export {
+ ContainerListResponsesPage,
+ Containers,
+ type ContainerCreateResponse,
+ type ContainerRetrieveResponse,
+ type ContainerListResponse,
+ type ContainerCreateParams,
+ type ContainerListParams,
+} from './containers/containers';
export {
Embeddings,
type CreateEmbeddingResponse,
@@ -33,10 +42,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +63,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..40dd9a6fb 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -37,7 +37,13 @@ export type ParsedResponseOutputItem<ParsedT> =
| ResponseFileSearchToolCall
| ResponseFunctionWebSearch
| ResponseComputerToolCall
- | ResponseReasoningItem;
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
export interface ParsedResponse<ParsedT> extends Response {
output: Array<ParsedResponseOutputItem<ParsedT>>;
@@ -61,6 +67,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +104,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +131,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -130,12 +158,33 @@ export class Responses extends APIResource {
/**
* Creates a model response stream
*/
+
stream<Params extends ResponseStreamParams, ParsedT = ExtractParsedContentFromParams<Params>>(
body: Params,
options?: Core.RequestOptions,
): ResponseStream<ParsedT> {
return ResponseStream.createResponse<ParsedT>(this._client, body, options);
}
+
+ /**
+ * Cancels a model response with the given ID. Only responses created with the
+ * `background` parameter set to `true` can be cancelled.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ *
+ * @example
+ * ```ts
+ * await client.responses.cancel(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
+ */
+
+ cancel(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.post(`/responses/${responseId}/cancel`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
}
export class ResponseItemsPage extends CursorPage<ResponseItem> {}
@@ -158,7 +207,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +258,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | null;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +307,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -392,6 +441,12 @@ export interface Response {
*/
top_p: number | null;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* An upper bound for the number of tokens that can be generated for a response,
* including visible output tokens and
@@ -437,7 +492,7 @@ export interface Response {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
status?: ResponseStatus;
@@ -496,6 +551,11 @@ export interface ResponseAudioDeltaEvent {
*/
delta: string;
+ /**
+ * A sequence number for this chunk of the stream response.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.delta`.
*/
@@ -506,6 +566,11 @@ export interface ResponseAudioDeltaEvent {
* Emitted when the audio response is complete.
*/
export interface ResponseAudioDoneEvent {
+ /**
+ * The sequence number of the delta.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.done`.
*/
@@ -521,6 +586,11 @@ export interface ResponseAudioTranscriptDeltaEvent {
*/
delta: string;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.transcript.delta`.
*/
@@ -531,6 +601,11 @@ export interface ResponseAudioTranscriptDeltaEvent {
* Emitted when the full audio transcript is completed.
*/
export interface ResponseAudioTranscriptDoneEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.transcript.done`.
*/
@@ -551,6 +626,11 @@ export interface ResponseCodeInterpreterCallCodeDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.code.delta`.
*/
@@ -571,6 +651,11 @@ export interface ResponseCodeInterpreterCallCodeDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.code.done`.
*/
@@ -591,6 +676,11 @@ export interface ResponseCodeInterpreterCallCompletedEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.completed`.
*/
@@ -611,6 +701,11 @@ export interface ResponseCodeInterpreterCallInProgressEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.in_progress`.
*/
@@ -631,6 +726,11 @@ export interface ResponseCodeInterpreterCallInterpretingEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.interpreting`.
*/
@@ -665,6 +765,11 @@ export interface ResponseCodeInterpreterToolCall {
* The type of the code interpreter tool call. Always `code_interpreter_call`.
*/
type: 'code_interpreter_call';
+
+ /**
+ * The ID of the container used to run the code.
+ */
+ container_id?: string;
}
export namespace ResponseCodeInterpreterToolCall {
@@ -719,6 +824,11 @@ export interface ResponseCompletedEvent {
*/
response: Response;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.completed`.
*/
@@ -1101,6 +1211,11 @@ export interface ResponseContentPartAddedEvent {
*/
part: ResponseOutputText | ResponseOutputRefusal;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.content_part.added`.
*/
@@ -1131,6 +1246,11 @@ export interface ResponseContentPartDoneEvent {
*/
part: ResponseOutputText | ResponseOutputRefusal;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.content_part.done`.
*/
@@ -1146,6 +1266,11 @@ export interface ResponseCreatedEvent {
*/
response: Response;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.created`.
*/
@@ -1204,6 +1329,11 @@ export interface ResponseErrorEvent {
*/
param: string | null;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `error`.
*/
@@ -1219,6 +1349,11 @@ export interface ResponseFailedEvent {
*/
response: Response;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.failed`.
*/
@@ -1239,6 +1374,11 @@ export interface ResponseFileSearchCallCompletedEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.file_search_call.completed`.
*/
@@ -1259,6 +1399,11 @@ export interface ResponseFileSearchCallInProgressEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.file_search_call.in_progress`.
*/
@@ -1279,6 +1424,11 @@ export interface ResponseFileSearchCallSearchingEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.file_search_call.searching`.
*/
@@ -1429,6 +1579,11 @@ export interface ResponseFunctionCallArgumentsDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.function_call_arguments.delta`.
*/
@@ -1454,6 +1609,11 @@ export interface ResponseFunctionCallArgumentsDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
type: 'response.function_call_arguments.done';
}
@@ -1557,6 +1717,119 @@ export interface ResponseFunctionWebSearch {
type: 'web_search_call';
}
+/**
+ * Emitted when an image generation tool call has completed and the final image is
+ * available.
+ */
+export interface ResponseImageGenCallCompletedEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.completed'.
+ */
+ type: 'response.image_generation_call.completed';
+}
+
+/**
+ * Emitted when an image generation tool call is actively generating an image
+ * (intermediate state).
+ */
+export interface ResponseImageGenCallGeneratingEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.generating'.
+ */
+ type: 'response.image_generation_call.generating';
+}
+
+/**
+ * Emitted when an image generation tool call is in progress.
+ */
+export interface ResponseImageGenCallInProgressEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.in_progress'.
+ */
+ type: 'response.image_generation_call.in_progress';
+}
+
+/**
+ * Emitted when a partial image is available during image generation streaming.
+ */
+export interface ResponseImageGenCallPartialImageEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * Base64-encoded partial image data, suitable for rendering as an image.
+ */
+ partial_image_b64: string;
+
+ /**
+ * 0-based index for the partial image (backend is 1-based, but this is 0-based for
+ * the user).
+ */
+ partial_image_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.partial_image'.
+ */
+ type: 'response.image_generation_call.partial_image';
+}
+
/**
* Emitted when the response is in progress.
*/
@@ -1566,6 +1839,11 @@ export interface ResponseInProgressEvent {
*/
response: Response;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.in_progress`.
*/
@@ -1581,11 +1859,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1596,6 +1880,11 @@ export interface ResponseIncompleteEvent {
*/
response: Response;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.incomplete`.
*/
@@ -1650,7 +1939,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1956,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1704,6 +1993,14 @@ export type ResponseInputItem =
| ResponseFunctionToolCall
| ResponseInputItem.FunctionCallOutput
| ResponseReasoningItem
+ | ResponseInputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseInputItem.LocalShellCall
+ | ResponseInputItem.LocalShellCallOutput
+ | ResponseInputItem.McpListTools
+ | ResponseInputItem.McpApprovalRequest
+ | ResponseInputItem.McpApprovalResponse
+ | ResponseInputItem.McpCall
| ResponseInputItem.ItemReference;
export namespace ResponseInputItem {
@@ -1758,19 +2055,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +2083,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,104 +2115,824 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
- * An internal identifier for an item to reference.
+ * An image generation request made by the model.
*/
- export interface ItemReference {
+ export interface ImageGenerationCall {
/**
- * The ID of the item to reference.
+ * The unique ID of the image generation call.
*/
id: string;
/**
- * The type of item to reference. Always `item_reference`.
+ * The generated image encoded in base64.
*/
- type: 'item_reference';
- }
-}
+ result: string | null;
-/**
- * A list of one or many input items to the model, containing different content
- * types.
- */
-export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
-export interface ResponseInputMessageItem {
- /**
- * The unique ID of the message input.
- */
- id: string;
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
/**
- * A list of one or many input items to the model, containing different content
- * types.
+ * A tool call to run a command on the local shell.
*/
- content: ResponseInputMessageContentList;
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
- /**
- * The role of the message input. One of `user`, `system`, or `developer`.
- */
- role: 'user' | 'system' | 'developer';
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
- /**
- * The status of item. One of `in_progress`, `completed`, or `incomplete`.
- * Populated when items are returned via API.
- */
- status?: 'in_progress' | 'completed' | 'incomplete';
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
- /**
- * The type of the message input. Always set to `message`.
- */
- type?: 'message';
-}
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
-/**
- * A text input to the model.
- */
-export interface ResponseInputText {
- /**
- * The text input to the model.
- */
- text: string;
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
- /**
- * The type of the input item. Always `input_text`.
- */
- type: 'input_text';
-}
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
-/**
- * Content item used to generate a response.
- */
-export type ResponseItem =
- | ResponseInputMessageItem
- | ResponseOutputMessage
- | ResponseFileSearchToolCall
- | ResponseComputerToolCall
- | ResponseComputerToolCallOutputItem
- | ResponseFunctionWebSearch
- | ResponseFunctionToolCallItem
- | ResponseFunctionToolCallOutputItem;
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
-/**
- * An audio output from the model.
- */
-export interface ResponseOutputAudio {
- /**
- * Base64-encoded audio data from the model.
- */
- data: string;
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
- /**
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * The output of a local shell tool call.
+ */
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
+
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
+
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * The unique ID of the approval response
+ */
+ id?: string | null;
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * An internal identifier for an item to reference.
+ */
+ export interface ItemReference {
+ /**
+ * The ID of the item to reference.
+ */
+ id: string;
+
+ /**
+ * The type of item to reference. Always `item_reference`.
+ */
+ type?: 'item_reference' | null;
+ }
+}
+
+/**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+
+export interface ResponseInputMessageItem {
+ /**
+ * The unique ID of the message input.
+ */
+ id: string;
+
+ /**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+ content: ResponseInputMessageContentList;
+
+ /**
+ * The role of the message input. One of `user`, `system`, or `developer`.
+ */
+ role: 'user' | 'system' | 'developer';
+
+ /**
+ * The status of item. One of `in_progress`, `completed`, or `incomplete`.
+ * Populated when items are returned via API.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the message input. Always set to `message`.
+ */
+ type?: 'message';
+}
+
+/**
+ * A text input to the model.
+ */
+export interface ResponseInputText {
+ /**
+ * The text input to the model.
+ */
+ text: string;
+
+ /**
+ * The type of the input item. Always `input_text`.
+ */
+ type: 'input_text';
+}
+
+/**
+ * Content item used to generate a response.
+ */
+export type ResponseItem =
+ | ResponseInputMessageItem
+ | ResponseOutputMessage
+ | ResponseFileSearchToolCall
+ | ResponseComputerToolCall
+ | ResponseComputerToolCallOutputItem
+ | ResponseFunctionWebSearch
+ | ResponseFunctionToolCallItem
+ | ResponseFunctionToolCallOutputItem
+ | ResponseItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseItem.LocalShellCall
+ | ResponseItem.LocalShellCallOutput
+ | ResponseItem.McpListTools
+ | ResponseItem.McpApprovalRequest
+ | ResponseItem.McpApprovalResponse
+ | ResponseItem.McpCall;
+
+export namespace ResponseItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * The output of a local shell tool call.
+ */
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
+
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
+
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The unique ID of the approval response
+ */
+ id: string;
+
+ /**
+ * The ID of the approval request being answered.
+ , */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the arguments of an MCP tool
+ * call.
+ */
+export interface ResponseMcpCallArgumentsDeltaEvent {
+ /**
+ * The partial update to the arguments for the MCP tool call.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_delta'.
+ */
+ type: 'response.mcp_call.arguments_delta';
+}
+
+/**
+ * Emitted when the arguments for an MCP tool call are finalized.
+ */
+export interface ResponseMcpCallArgumentsDoneEvent {
+ /**
+ * The finalized arguments for the MCP tool call.
+ */
+ arguments: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_done'.
+ */
+ type: 'response.mcp_call.arguments_done';
+}
+
+/**
+ * Emitted when an MCP tool call has completed successfully.
+ */
+export interface ResponseMcpCallCompletedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.completed'.
+ */
+ type: 'response.mcp_call.completed';
+}
+
+/**
+ * Emitted when an MCP tool call has failed.
+ */
+export interface ResponseMcpCallFailedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.failed'.
+ */
+ type: 'response.mcp_call.failed';
+}
+
+/**
+ * Emitted when an MCP tool call is in progress.
+ */
+export interface ResponseMcpCallInProgressEvent {
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.in_progress'.
+ */
+ type: 'response.mcp_call.in_progress';
+}
+
+/**
+ * Emitted when the list of available MCP tools has been successfully retrieved.
+ */
+export interface ResponseMcpListToolsCompletedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.completed'.
+ */
+ type: 'response.mcp_list_tools.completed';
+}
+
+/**
+ * Emitted when the attempt to list available MCP tools has failed.
+ */
+export interface ResponseMcpListToolsFailedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.failed'.
+ */
+ type: 'response.mcp_list_tools.failed';
+}
+
+/**
+ * Emitted when the system is in the process of retrieving the list of available
+ * MCP tools.
+ */
+export interface ResponseMcpListToolsInProgressEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.in_progress'.
+ */
+ type: 'response.mcp_list_tools.in_progress';
+}
+
+/**
+ * An audio output from the model.
+ */
+export interface ResponseOutputAudio {
+ /**
+ * Base64-encoded audio data from the model.
+ */
+ data: string;
+
+ /**
* The transcript of the audio data from the model.
*/
transcript: string;
@@ -1935,7 +2952,234 @@ export type ResponseOutputItem =
| ResponseFunctionToolCall
| ResponseFunctionWebSearch
| ResponseComputerToolCall
- | ResponseReasoningItem;
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
+
+export namespace ResponseOutputItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+}
/**
* Emitted when a new output item is added.
@@ -1951,6 +3195,11 @@ export interface ResponseOutputItemAddedEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_item.added`.
*/
@@ -1971,6 +3220,11 @@ export interface ResponseOutputItemDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_item.done`.
*/
@@ -2075,51 +3329,183 @@ export namespace ResponseOutputText {
*/
end_index: number;
- /**
- * The index of the first character of the URL citation in the message.
- */
- start_index: number;
+ /**
+ * The index of the first character of the URL citation in the message.
+ */
+ start_index: number;
+
+ /**
+ * The title of the web resource.
+ */
+ title: string;
+
+ /**
+ * The type of the URL citation. Always `url_citation`.
+ */
+ type: 'url_citation';
+
+ /**
+ * The URL of the web resource.
+ */
+ url: string;
+ }
+
+ /**
+ * A path to a file.
+ */
+ export interface FilePath {
+ /**
+ * The ID of the file.
+ */
+ file_id: string;
+
+ /**
+ * The index of the file in the list of files.
+ */
+ index: number;
+
+ /**
+ * The type of the file path. Always `file_path`.
+ */
+ type: 'file_path';
+ }
+}
+
+/**
+ * Emitted when an annotation is added to output text content.
+ */
+export interface ResponseOutputTextAnnotationAddedEvent {
+ /**
+ * The annotation object being added. (See annotation schema for details.)
+ */
+ annotation: unknown;
+
+ /**
+ * The index of the annotation within the content part.
+ */
+ annotation_index: number;
+
+ /**
+ * The index of the content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The unique identifier of the item to which the annotation is being added.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.output_text_annotation.added'.
+ */
+ type: 'response.output_text_annotation.added';
+}
+
+/**
+ * Emitted when a response is queued and waiting to be processed.
+ */
+export interface ResponseQueuedEvent {
+ /**
+ * The full response object that is queued.
+ */
+ response: Response;
+
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.queued'.
+ */
+ type: 'response.queued';
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the reasoning content.
+ */
+export interface ResponseReasoningDeltaEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The partial update to the reasoning content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which reasoning is being updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning.delta'.
+ */
+ type: 'response.reasoning.delta';
+}
- /**
- * The title of the web resource.
- */
- title: string;
+/**
+ * Emitted when the reasoning content is finalized for an item.
+ */
+export interface ResponseReasoningDoneEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
- /**
- * The type of the URL citation. Always `url_citation`.
- */
- type: 'url_citation';
+ /**
+ * The unique identifier of the item for which reasoning is finalized.
+ */
+ item_id: string;
- /**
- * The URL of the web resource.
- */
- url: string;
- }
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
/**
- * A path to a file.
+ * The sequence number of this event.
*/
- export interface FilePath {
- /**
- * The ID of the file.
- */
- file_id: string;
+ sequence_number: number;
- /**
- * The index of the file in the list of files.
- */
- index: number;
+ /**
+ * The finalized reasoning text.
+ */
+ text: string;
- /**
- * The type of the file path. Always `file_path`.
- */
- type: 'file_path';
- }
+ /**
+ * The type of the event. Always 'response.reasoning.done'.
+ */
+ type: 'response.reasoning.done';
}
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +3523,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2158,6 +3550,77 @@ export namespace ResponseReasoningItem {
}
}
+/**
+ * Emitted when there is a delta (partial update) to the reasoning summary content.
+ */
+export interface ResponseReasoningSummaryDeltaEvent {
+ /**
+ * The partial update to the reasoning summary content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which the reasoning summary is being
+ * updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.delta'.
+ */
+ type: 'response.reasoning_summary.delta';
+}
+
+/**
+ * Emitted when the reasoning summary content is finalized for an item.
+ */
+export interface ResponseReasoningSummaryDoneEvent {
+ /**
+ * The unique identifier of the item for which the reasoning summary is finalized.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The finalized reasoning summary text.
+ */
+ text: string;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.done'.
+ */
+ type: 'response.reasoning_summary.done';
+}
+
/**
* Emitted when a new reasoning summary part is added.
*/
@@ -2177,6 +3640,11 @@ export interface ResponseReasoningSummaryPartAddedEvent {
*/
part: ResponseReasoningSummaryPartAddedEvent.Part;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2224,6 +3692,11 @@ export interface ResponseReasoningSummaryPartDoneEvent {
*/
part: ResponseReasoningSummaryPartDoneEvent.Part;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2271,6 +3744,11 @@ export interface ResponseReasoningSummaryTextDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2296,6 +3774,11 @@ export interface ResponseReasoningSummaryTextDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2336,6 +3819,11 @@ export interface ResponseRefusalDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.refusal.delta`.
*/
@@ -2366,6 +3854,11 @@ export interface ResponseRefusalDoneEvent {
*/
refusal: string;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.refusal.done`.
*/
@@ -2374,9 +3867,9 @@ export interface ResponseRefusalDoneEvent {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
-export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'incomplete';
+export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'cancelled' | 'queued' | 'incomplete';
/**
* Emitted when there is a partial audio response.
@@ -2417,7 +3910,25 @@ export type ResponseStreamEvent =
| ResponseTextDoneEvent
| ResponseWebSearchCallCompletedEvent
| ResponseWebSearchCallInProgressEvent
- | ResponseWebSearchCallSearchingEvent;
+ | ResponseWebSearchCallSearchingEvent
+ | ResponseImageGenCallCompletedEvent
+ | ResponseImageGenCallGeneratingEvent
+ | ResponseImageGenCallInProgressEvent
+ | ResponseImageGenCallPartialImageEvent
+ | ResponseMcpCallArgumentsDeltaEvent
+ | ResponseMcpCallArgumentsDoneEvent
+ | ResponseMcpCallCompletedEvent
+ | ResponseMcpCallFailedEvent
+ | ResponseMcpCallInProgressEvent
+ | ResponseMcpListToolsCompletedEvent
+ | ResponseMcpListToolsFailedEvent
+ | ResponseMcpListToolsInProgressEvent
+ | ResponseOutputTextAnnotationAddedEvent
+ | ResponseQueuedEvent
+ | ResponseReasoningDeltaEvent
+ | ResponseReasoningDoneEvent
+ | ResponseReasoningSummaryDeltaEvent
+ | ResponseReasoningSummaryDoneEven,t;
/**
* Emitted when a text annotation is added.
@@ -2451,6 +3962,11 @@ export interface ResponseTextAnnotationDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_text.annotation.added`.
*/
@@ -2579,6 +4095,11 @@ export interface ResponseTextDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_text.delta`.
*/
@@ -2604,6 +4125,11 @@ export interface ResponseTextDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The text content that is finalized.
*/
@@ -2730,11 +4256,231 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool =
+ | FunctionTool
+ | FileSearchTool
+ | WebSearchTool
+ | ComputerTool
+ | Tool.Mcp
+ | Tool.CodeInterpreter
+ | Tool.ImageGeneration
+ | Tool.LocalShell;
+
+export namespace Tool {
+ /**
+ * Give the model access to additional tools via remote Model Context Protocol
+ * (MCP) servers.
+ * [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp).
+ */
+ export interface Mcp {
+ /**
+ * A label for this MCP server, used to identify it in tool calls.
+ */
+ server_label: string;
+
+ /**
+ * The URL for the MCP server.
+ */
+ server_url: string;
+
+ /**
+ * The type of the MCP tool. Always `mcp`.
+ */
+ type: 'mcp';
+
+ /**
+ * List of allowed tool names or a filter object.
+ */
+ allowed_tools?: Array<string> | Mcp.McpAllowedToolsFilter | null;
+
+ /**
+ * Optional HTTP headers to send to the MCP server. Use for authentication or other
+ * purposes.
+ */
+ headers?: Record<string, string> | null;
+
+ /**
+ * Specify which of the MCP server's tools require approval.
+ */
+ require_approval?: Mcp.McpToolApprovalFilter | 'always' | 'never' | null;
+ }
+
+ export namespace Mcp {
+ /**
+ * A filter object to specify which tools are allowed.
+ */
+ export interface McpAllowedToolsFilter {
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export interface McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ always?: McpToolApprovalFilter.Always;
+
+ /**
+ * A list of tools that never require approval.
+ */
+ never?: McpToolApprovalFilter.Never;
+
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export namespace McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ export interface Always {
+ /**
+ * List of tools that require approval.
+ */
+ tool_names?: Array<string>;
+ }
+
+ /**
+ * A list of tools that never require approval.
+ */
+ export interface Never {
+ /**
+ * List of tools that do not require approval.
+ */
+ tool_names?: Array<string>;
+ }
+ }
+ }
+
+ /**
+ * A tool that runs Python code to help generate a response to a prompt.
+ */
+ export interface CodeInterpreter {
+ /**
+ * The code interpreter container. Can be a container ID or an object that
+ * specifies uploaded file IDs to make available to your code.
+ */
+ container: string | CodeInterpreter.CodeInterpreterToolAuto;
+
+ /**
+ * The type of the code interpreter tool. Always `code_interpreter`.
+ */
+ type: 'code_interpreter';
+ }
+
+ export namespace CodeInterpreter {
+ /**
+ * Configuration for a code interpreter container. Optionally specify the IDs of
+ * the files to run the code on.
+ */
+ export interface CodeInterpreterToolAuto {
+ /**
+ * Always `auto`.
+ */
+ type: 'auto';
+
+ /**
+ * An optional list of uploaded files to make available to your code.
+ */
+ file_ids?: Array<string>;
+ }
+ }
+
+ /**
+ * A tool that generates images using a model like `gpt-image-1`.
+ */
+ export interface ImageGeneration {
+ /**
+ * The type of the image generation tool. Always `image_generation`.
+ */
+ type: 'image_generation';
+
+ /**
+ * Background type for the generated image. One of `transparent`, `opaque`, or
+ * `auto`. Default: `auto`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto';
+
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ input_image_mask?: ImageGeneration.InputImageMask;
+
+ /**
+ * The image generation model to use. Default: `gpt-image-1`.
+ */
+ model?: 'gpt-image-1';
+
+ /**
+ * Moderation level for the generated image. Default: `auto`.
+ */
+ moderation?: 'auto' | 'low';
+
+ /**
+ * Compression level for the output image. Default: 100.
+ */
+ output_compression?: number;
+
+ /**
+ * The output format of the generated image. One of `png`, `webp`, or `jpeg`.
+ * Default: `png`.
+ */
+ output_format?: 'png' | 'webp' | 'jpeg';
+
+ /**
+ * Number of partial images to generate in streaming mode, from 0 (default value)
+ * to 3.
+ */
+ partial_images?: number;
+
+ /**
+ * The quality of the generated image. One of `low`, `medium`, `high`, or `auto`.
+ * Default: `auto`.
+ */
+ quality?: 'low' | 'medium' | 'high' | 'auto';
+
+ /**
+ * The size of the generated image. One of `1024x1024`, `1024x1536`, `1536x1024`,
+ * or `auto`. Default: `auto`.
+ */
+ size?: '1024x1024' | '1024x1536' | '1536x1024' | 'auto';
+ }
+
+ export namespace ImageGeneration {
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ export interface InputImageMask {
+ /**
+ * File ID for the mask image.
+ */
+ file_id?: string;
+
+ /**
+ * Base64-encoded mask image.
+ */
+ image_url?: string;
+ }
+ }
+
+ /**
+ * A tool that allows the model to execute shell commands in a local environment.
+ */
+ export interface LocalShell {
+ /**
+ * The type of the local shell tool. Always `local_shell`.
+ */
+ type: 'local_shell';
+ }
+}
/**
* Use this option to force the model to call a specific function.
@@ -2777,8 +4523,18 @@ export interface ToolChoiceTypes {
* - `file_search`
* - `web_search_preview`
* - `computer_use_preview`
- */
- type: 'file_search' | 'web_search_preview' | 'computer_use_preview' | 'web_search_preview_2025_03_11';
+ * - `code_interpreter`
+ * - `mcp`
+ * - `image_generation`
+ */
+ type:
+ | 'file_search'
+ | 'web_search_preview'
+ | 'computer_use_preview'
+ | 'web_search_preview_2025_03_11'
+ | 'image_generation'
+ | 'code_interpreter'
+ | 'mcp';
}
/**
@@ -2788,10 +4544,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +4555,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +4574,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2860,6 +4620,12 @@ export interface ResponseCreateParamsBase {
*/
model: Shared.ResponsesModel;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* Specify additional output data to include in the model response. Currently
* supported values are:
@@ -2869,6 +4635,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
@@ -3104,6 +4875,10 @@ export declare namespace Responses {
type ResponseFunctionToolCallItem as ResponseFunctionToolCallItem,
type ResponseFunctionToolCallOutputItem as ResponseFunctionToolCallOutputItem,
type ResponseFunctionWebSearch as ResponseFunctionWebSearch,
+ type ResponseImageGenCallCompletedEvent as ResponseImageGenCallCompletedEvent,
+ type ResponseImageGenCallGeneratingEvent as ResponseImageGenCallGeneratingEvent,
+ type ResponseImageGenCallInProgressEvent as ResponseImageGenCallInProgressEvent,
+ type ResponseImageGenCallPartialImageEvent as ResponseImageGenCallPartialImageEvent,
type ResponseInProgressEvent as ResponseInProgressEvent,
type ResponseIncludable as ResponseIncludable,
type ResponseIncompleteEvent as ResponseIncompleteEvent,
@@ -3117,6 +4892,14 @@ export declare namespace Responses {
type ResponseInputMessageItem as ResponseInputMessageItem,
type ResponseInputText as ResponseInputText,
type ResponseItem as ResponseItem,
+ type ResponseMcpCallArgumentsDeltaEvent as ResponseMcpCallArgumentsDeltaEvent,
+ type ResponseMcpCallArgumentsDoneEvent as ResponseMcpCallArgumentsDoneEvent,
+ type ResponseMcpCallCompletedEvent as ResponseMcpCallCompletedEvent,
+ type ResponseMcpCallFailedEvent as ResponseMcpCallFailedEvent,
+ type ResponseMcpCallInProgressEvent as ResponseMcpCallInProgressEvent,
+ type ResponseMcpListToolsCompletedEvent as ResponseMcpListToolsCompletedEvent,
+ type ResponseMcpListToolsFailedEvent as ResponseMcpListToolsFailedEvent,
+ type ResponseMcpListToolsInProgressEvent as ResponseMcpListToolsInProgressEvent,
type ResponseOutputAudio as ResponseOutputAudio,
type ResponseOutputItem as ResponseOutputItem,
type ResponseOutputItemAddedEvent as ResponseOutputItemAddedEvent,
@@ -3124,7 +4907,13 @@ export declare namespace Responses {
type ResponseOutputMessage as ResponseOutputMessage,
type ResponseOutputRefusal as ResponseOutputRefusal,
type ResponseOutputText as ResponseOutputText,
+ type ResponseOutputTextAnnotationAddedEvent as ResponseOutputTextAnnotationAddedEvent,
+ type ResponseQueuedEvent as ResponseQueuedEvent,
+ type ResponseReasoningDeltaEvent as ResponseReasoningDeltaEvent,
+ type ResponseReasoningDoneEvent as ResponseReasoningDoneEvent,
type ResponseReasoningItem as ResponseReasoningItem,
+ type ResponseReasoningSummaryDeltaEvent as ResponseReasoningSummaryDeltaEvent,
+ type ResponseReasoningSummaryDoneEvent as ResponseReasoningSummaryDoneEvent,
type ResponseReasoningSummaryPartAddedEvent as ResponseReasoningSummaryPartAddedEvent,
type ResponseReasoningSummaryPartDoneEvent as ResponseReasoningSummaryPartDoneEvent,
type ResponseReasoningSummaryTextDeltaEvent as ResponseReasoningSummaryTextDeltaEvent,
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..180cea6da 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.102.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/containers/containers.test.ts b/tests/api-resources/containers/containers.test.ts
new file mode 100644
index 000000000..11c6f166c
--- /dev/null
+++ tests/api-resources/containers/containers.test.ts
@@ -0,0 +1,94 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource containers', () => {
+ test('create: only required params', async () => {
+ const responsePromise = client.containers.create({ name: 'name' });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('create: required and optional params', async () => {
+ const response = await client.containers.create({
+ name: 'name',
+ expires_after: { anchor: 'last_active_at', minutes: 0 },
+ file_ids: ['string'],
+ });
+ });
+
+ test('retrieve', async () => {
+ const responsePromise = client.containers.retrieve('container_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.retrieve('container_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('list', async () => {
+ const responsePromise = client.containers.list();
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('list: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(client.containers.list({ path: '/_stainless_unknown_path' })).rejects.toThrow(
+ OpenAI.NotFoundError,
+ );
+ });
+
+ test('list: request options and params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.list(
+ { after: 'after', limit: 0, order: 'asc' },
+ { path: '/_stainless_unknown_path' },
+ ),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('del', async () => {
+ const responsePromise = client.containers.del('container_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('del: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(client.containers.del('container_id', { path: '/_stainless_unknown_path' })).rejects.toThrow(
+ OpenAI.NotFoundError,
+ );
+ });
+});
diff --git a/tests/api-resources/containers/files/content.test.ts b/tests/api-resources/containers/files/content.test.ts
new file mode 100644
index 000000000..9215c001a
--- /dev/null
+++ tests/api-resources/containers/files/content.test.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource content', () => {
+ test('retrieve', async () => {
+ const responsePromise = client.containers.files.content.retrieve('container_id', 'file_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.content.retrieve('container_id', 'file_id', {
+ path: '/_stainless_unknown_path',
+ }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+});
diff --git a/tests/api-resources/containers/files/files.test.ts b/tests/api-resources/containers/files/files.test.ts
new file mode 100644
index 000000000..79f47e9ad
--- /dev/null
+++ tests/api-resources/containers/files/files.test.ts
@@ -0,0 +1,87 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource files', () => {
+ test('create', async () => {
+ const responsePromise = client.containers.files.create('container_id', {});
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve', async () => {
+ const responsePromise = client.containers.files.retrieve('container_id', 'file_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.retrieve('container_id', 'file_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('list', async () => {
+ const responsePromise = client.containers.files.list('container_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('list: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.list('container_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('list: request options and params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.list(
+ 'container_id',
+ { after: 'after', limit: 0, order: 'asc' },
+ { path: '/_stainless_unknown_path' },
+ ),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('del', async () => {
+ const responsePromise = client.containers.files.del('container_id', 'file_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('del: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.del('container_id', 'file_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+});
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
diff --git tests/api-resources/responses/responses.test.ts tests/api-resources/responses/responses.test.ts
index cf7e9cf3c..f8e653c52 100644
--- tests/api-resources/responses/responses.test.ts
+++ tests/api-resources/responses/responses.test.ts
@@ -24,6 +24,7 @@ describe('resource responses', () => {
const response = await client.responses.create({
input: 'string',
model: 'gpt-4o',
+ background: true,
include: ['file_search_call.results'],
instructions: 'instructions',
max_output_tokens: 0,
@@ -39,11 +40,11 @@ describe('resource responses', () => {
tool_choice: 'none',
tools: [
{
- type: 'file_search',
- vector_store_ids: ['string'],
- filters: { key: 'key', type: 'eq', value: 'string' },
- max_num_results: 0,
- ranking_options: { ranker: 'auto', score_threshold: 0 },
+ name: 'name',
+ parameters: { foo: 'bar' },
+ strict: true,
+ type: 'function',
+ description: 'description',
},
],
top_p: 1,
@@ -100,4 +101,22 @@ describe('resource responses', () => {
client.responses.del('resp_677efb5139a88190b512bc3fef8e535d', { path: '/_stainless_unknown_path' }),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('cancel', async () => {
+ const responsePromise = client.responses.cancel('resp_677efb5139a88190b512bc3fef8e535d');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('cancel: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.responses.cancel('resp_677efb5139a88190b512bc3fef8e535d', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
Description
This PR updates the OpenAI Node.js SDK from version 4.96.2 to 4.102.0. The changes include upgrading from Node.js 18 to Node.js 20 in CI workflows, adding support for new APIs (including Containers API, reinforcement fine-tuning), introducing new models, fixing documentation typos, and enhancing existing API interfaces with new parameters and capabilities.
Changes
Changes
CI/Workflow Updates
- Updated Node.js version from 18 to 20 in all GitHub workflow files (ci.yml, publish-jsr.yml, publish-npm.yml)
Version & Configuration Updates
- Bumped version from 4.96.2 to 4.102.0 in package.json, jsr.json, and related version files
- Updated API statistics and OpenAPI specs in .stats.yml
- Added CHANGELOG entries for versions 4.97.0 through 4.102.0
Documentation Improvements
- Fixed typos and grammar in README.md and SECURITY.md
- Enhanced JSDoc comments with examples for many API methods
- Updated API documentation in api.md with new endpoints and parameters
New APIs and Features
- Added Containers API for managing file containers
- Added support for reinforcement fine-tuning
- Added new grader models and tools for evaluations
- Extended Responses API with new capabilities (image generation, MCP)
- Added support for the
backgroundparameter in response creation - Added new audio transcription features including chunking strategies
Dependency & Interface Updates
- Added new model types in shared.ts
- Extended existing interfaces with new parameters
- Added new event types for streaming responses
- Enhanced error handling and status reporting
sequenceDiagram
title OpenAI Node.js SDK v4.102.0 Update
Client->>+OpenAI: Update Node.js from v18 to v20
Note over Client,OpenAI: CI workflow changes
OpenAI-->>-Client: Builds with Node.js 20
Client->>+OpenAI: Use Containers API
OpenAI-->>-Client: Container management functionality
Client->>+OpenAI: Initialize new fine-tuning with reinforcement
OpenAI-->>-Client: Advanced fine-tuning options
Client->>+OpenAI: Create responses with background parameter
OpenAI-->>-Client: Background processing support
Client->>+OpenAI: Stream response events
Note over Client,OpenAI: New event types and streaming capabilities
OpenAI-->>Client: Text delta events
OpenAI-->>Client: Image generation events
OpenAI-->>Client: MCP events
OpenAI-->>-Client: Complete response
Client->>+OpenAI: Use transcription with chunking strategy
OpenAI-->>-Client: Enhanced audio processing
bedrock debug - [puLL-Merge] - openai/[email protected]
Diff
diff --git .github/workflows/ci.yml .github/workflows/ci.yml
index 49a043930..09f1636b6 100644
--- .github/workflows/ci.yml
+++ .github/workflows/ci.yml
@@ -72,7 +72,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Bootstrap
run: ./scripts/bootstrap
@@ -92,7 +92,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v4
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
yarn install
diff --git .github/workflows/publish-jsr.yml .github/workflows/publish-jsr.yml
index 1e46d6bfb..e74673c1f 100644
--- .github/workflows/publish-jsr.yml
+++ .github/workflows/publish-jsr.yml
@@ -19,7 +19,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .github/workflows/publish-npm.yml .github/workflows/publish-npm.yml
index 5a3711b53..0662a79c5 100644
--- .github/workflows/publish-npm.yml
+++ .github/workflows/publish-npm.yml
@@ -16,7 +16,7 @@ jobs:
- name: Set up Node
uses: actions/setup-node@v3
with:
- node-version: '18'
+ node-version: '20'
- name: Install dependencies
run: |
diff --git .release-please-manifest.json .release-please-manifest.json
index 88f780d30..a39d68281 100644
--- .release-please-manifest.json
+++ .release-please-manifest.json
@@ -1,3 +1,3 @@
{
- ".": "4.96.2"
+ ".": "4.102.0"
}
diff --git .stats.yml .stats.yml
index d92408173..41319e5e5 100644
--- .stats.yml
+++ .stats.yml
@@ -1,4 +1,4 @@
-configured_endpoints: 97
-openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-8b68ae6b807dca92e914da1dd9e835a20f69b075e79102a264367fd7fddddb33.yml
-openapi_spec_hash: b6ade5b1a6327339e6669e1134de2d03
-config_hash: b597cd9a31e9e5ec709e2eefb4c54122
+configured_endpoints: 111
+openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-6af14840a810139bf407013167ce1c8fb21b6ef8eb0cc3db58b51af7d52c4b5a.yml
+openapi_spec_hash: 3241bde6b273cfec0035e522bd07985d
+config_hash: 7367b68a4e7db36885c1a886f57b17f6
diff --git CHANGELOG.md CHANGELOG.md
index d724d8922..b6254488a 100644
--- CHANGELOG.md
+++ CHANGELOG.md
@@ -1,5 +1,88 @@
# Changelog
+## 4.102.0 (2025-05-21)
+
+Full Changelog: [v4.101.0...v4.102.0](https://github.com/openai/openai-node/compare/v4.101.0...v4.102.0)
+
+### Features
+
+* **api:** add container endpoint ([e973476](https://github.com/openai/openai-node/commit/e9734764625275c50ef612ff934804be8cb2adff))
+
+## 4.101.0 (2025-05-21)
+
+Full Changelog: [v4.100.0...v4.101.0](https://github.com/openai/openai-node/compare/v4.100.0...v4.101.0)
+
+### Features
+
+* **api:** new API tools ([fb4014f](https://github.com/openai/openai-node/commit/fb4014ffac7b220d37bd03c94fa745386b010bf0))
+
+
+### Chores
+
+* **docs:** grammar improvements ([7761cfb](https://github.com/openai/openai-node/commit/7761cfb0a8a56d056a73c046a6a613f66ada4abe))
+* **internal:** version bump ([b40e830](https://github.com/openai/openai-node/commit/b40e8302ec11683b6a360a050236ad1284afc760))
+
+## 4.100.0 (2025-05-16)
+
+Full Changelog: [v4.99.0...v4.100.0](https://github.com/openai/openai-node/compare/v4.99.0...v4.100.0)
+
+### Features
+
+* **api:** further updates for evals API ([3f6f248](https://github.com/openai/openai-node/commit/3f6f248191b45015924be76fd5154d149c4ed8a0))
+
+
+### Chores
+
+* **internal:** version bump ([5123fe0](https://github.com/openai/openai-node/commit/5123fe08a56f3d0040b1cc67129382f3eacc3cca))
+
+## 4.99.0 (2025-05-16)
+
+Full Changelog: [v4.98.0...v4.99.0](https://github.com/openai/openai-node/compare/v4.98.0...v4.99.0)
+
+### Features
+
+* **api:** manual updates ([75eb804](https://github.com/openai/openai-node/commit/75eb804edd6ad653eaa22d47f8c6d09ee845ebf4))
+* **api:** responses x eval api ([5029f1a](https://github.com/openai/openai-node/commit/5029f1a05eb1e8601ada06e0a5ba49f4c2b83c02))
+* **api:** Updating Assistants and Evals API schemas ([27fd517](https://github.com/openai/openai-node/commit/27fd5173b20f75debe96024ae8f1ce58a8254d26))
+
+## 4.98.0 (2025-05-08)
+
+Full Changelog: [v4.97.0...v4.98.0](https://github.com/openai/openai-node/compare/v4.97.0...v4.98.0)
+
+### Features
+
+* **api:** Add reinforcement fine-tuning api support ([4aa7a79](https://github.com/openai/openai-node/commit/4aa7a7954c63caa26cc1640ace56093fe1cafa04))
+
+
+### Chores
+
+* **ci:** bump node version for release workflows ([2961f63](https://github.com/openai/openai-node/commit/2961f63c4d5b8ae8efdf8ea6581aa83c6b0f722e))
+* **internal:** fix formatting ([91a44fe](https://github.com/openai/openai-node/commit/91a44fe11c0847dc50d48a03a8d409ac4bece37a))
+
+
+### Documentation
+
+* add examples to tsdocs ([7d841b7](https://github.com/openai/openai-node/commit/7d841b7f98eb542a398fb9de12056125e8d6cb22))
+
+## 4.97.0 (2025-05-02)
+
+Full Changelog: [v4.96.2...v4.97.0](https://github.com/openai/openai-node/compare/v4.96.2...v4.97.0)
+
+### Features
+
+* **api:** add image sizes, reasoning encryption ([9c2113a](https://github.com/openai/openai-node/commit/9c2113af7c7ea9a797a0e39d07d9ad8627c96acb))
+
+
+### Chores
+
+* **docs:** add missing deprecation warnings ([253392c](https://github.com/openai/openai-node/commit/253392c93adca88e0ee83f784183b2128ff64a16))
+
+
+### Documentation
+
+* fix "procesing" -> "processing" in realtime examples ([#1406](https://github.com/openai/openai-node/issues/1406)) ([8717b9f](https://github.com/openai/openai-node/commit/8717b9fce87d03e51d40ee58f5d6259408405e1f))
+* **readme:** fix typo ([cab3478](https://github.com/openai/openai-node/commit/cab3478f195f9de5c21033a1b3684f52ad347ffc))
+
## 4.96.2 (2025-04-29)
Full Changelog: [v4.96.1...v4.96.2](https://github.com/openai/openai-node/compare/v4.96.1...v4.96.2)
diff --git README.md README.md
index 8515c81ed..bbf72226a 100644
--- README.md
+++ README.md
@@ -151,7 +151,7 @@ async function main() {
main();
```
-Error codes are as followed:
+Error codes are as follows:
| Status Code | Error Type |
| ----------- | -------------------------- |
diff --git SECURITY.md SECURITY.md
index 3b3bd8a66..4adb0c54f 100644
--- SECURITY.md
+++ SECURITY.md
@@ -16,13 +16,13 @@ before making any information public.
## Reporting Non-SDK Related Security Issues
If you encounter security issues that are not directly related to SDKs but pertain to the services
-or products provided by OpenAI please follow the respective company's security reporting guidelines.
+or products provided by OpenAI, please follow the respective company's security reporting guidelines.
### OpenAI Terms and Policies
Our Security Policy can be found at [Security Policy URL](https://openai.com/policies/coordinated-vulnerability-disclosure-policy).
-Please contact [email protected] for any questions or concerns regarding security of our services.
+Please contact [email protected] for any questions or concerns regarding the security of our services.
---
diff --git api.md api.md
index 49e6548a8..4e8187d0f 100644
--- api.md
+++ api.md
@@ -207,6 +207,17 @@ Methods:
# FineTuning
+## Methods
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">DpoMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">ReinforcementMethod</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedHyperparameters</a></code>
+- <code><a href="./src/resources/fine-tuning/methods.ts">SupervisedMethod</a></code>
+
## Jobs
Types:
@@ -224,6 +235,8 @@ Methods:
- <code title="get /fine_tuning/jobs">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">list</a>({ ...params }) -> FineTuningJobsPage</code>
- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/cancel">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">cancel</a>(fineTuningJobId) -> FineTuningJob</code>
- <code title="get /fine_tuning/jobs/{fine_tuning_job_id}/events">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">listEvents</a>(fineTuningJobId, { ...params }) -> FineTuningJobEventsPage</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/pause">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">pause</a>(fineTuningJobId) -> FineTuningJob</code>
+- <code title="post /fine_tuning/jobs/{fine_tuning_job_id}/resume">client.fineTuning.jobs.<a href="./src/resources/fine-tuning/jobs/jobs.ts">resume</a>(fineTuningJobId) -> FineTuningJob</code>
### Checkpoints
@@ -251,6 +264,33 @@ Methods:
- <code title="get /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">retrieve</a>(fineTunedModelCheckpoint, { ...params }) -> PermissionRetrieveResponse</code>
- <code title="delete /fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}">client.fineTuning.checkpoints.permissions.<a href="./src/resources/fine-tuning/checkpoints/permissions.ts">del</a>(fineTunedModelCheckpoint, permissionId) -> PermissionDeleteResponse</code>
+## Alpha
+
+### Graders
+
+Types:
+
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderRunResponse</a></code>
+- <code><a href="./src/resources/fine-tuning/alpha/graders.ts">GraderValidateResponse</a></code>
+
+Methods:
+
+- <code title="post /fine_tuning/alpha/graders/run">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">run</a>({ ...params }) -> GraderRunResponse</code>
+- <code title="post /fine_tuning/alpha/graders/validate">client.fineTuning.alpha.graders.<a href="./src/resources/fine-tuning/alpha/graders.ts">validate</a>({ ...params }) -> GraderValidateResponse</code>
+
+# Graders
+
+## GraderModels
+
+Types:
+
+- <code><a href="./src/resources/graders/grader-models.ts">LabelModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">MultiGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">PythonGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">ScoreModelGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">StringCheckGrader</a></code>
+- <code><a href="./src/resources/graders/grader-models.ts">TextSimilarityGrader</a></code>
+
# VectorStores
Types:
@@ -605,6 +645,10 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionToolCallOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseFunctionWebSearch</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallGeneratingEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseImageGenCallPartialImageEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncludable</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseIncompleteEvent</a></code>
@@ -618,6 +662,14 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseInputMessageItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseInputText</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallArgumentsDoneEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpCallInProgressEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsCompletedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsFailedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseMcpListToolsInProgressEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputAudio</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItem</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputItemAddedEvent</a></code>
@@ -625,7 +677,13 @@ Types:
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputMessage</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputRefusal</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseOutputText</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseOutputTextAnnotationAddedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseQueuedEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningItem</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDeltaEvent</a></code>
+- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartAddedEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryPartDoneEvent</a></code>
- <code><a href="./src/resources/responses/responses.ts">ResponseReasoningSummaryTextDeltaEvent</a></code>
@@ -653,6 +711,7 @@ Methods:
- <code title="post /responses">client.responses.<a href="./src/resources/responses/responses.ts">create</a>({ ...params }) -> Response</code>
- <code title="get /responses/{response_id}">client.responses.<a href="./src/resources/responses/responses.ts">retrieve</a>(responseId, { ...params }) -> Response</code>
- <code title="delete /responses/{response_id}">client.responses.<a href="./src/resources/responses/responses.ts">del</a>(responseId) -> void</code>
+- <code title="post /responses/{response_id}/cancel">client.responses.<a href="./src/resources/responses/responses.ts">cancel</a>(responseId) -> void</code>
## InputItems
@@ -669,10 +728,7 @@ Methods:
Types:
- <code><a href="./src/resources/evals/evals.ts">EvalCustomDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalLabelModelGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalStoredCompletionsDataSourceConfig</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalStringCheckGrader</a></code>
-- <code><a href="./src/resources/evals/evals.ts">EvalTextSimilarityGrader</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalCreateResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalRetrieveResponse</a></code>
- <code><a href="./src/resources/evals/evals.ts">EvalUpdateResponse</a></code>
@@ -719,3 +775,39 @@ Methods:
- <code title="get /evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}">client.evals.runs.outputItems.<a href="./src/resources/evals/runs/output-items.ts">retrieve</a>(evalId, runId, outputItemId) -> OutputItemRetrieveResponse</code>
- <code title="get /evals/{eval_id}/runs/{run_id}/output_items">client.evals.runs.outputItems.<a href="./src/resources/evals/runs/output-items.ts">list</a>(evalId, runId, { ...params }) -> OutputItemListResponsesPage</code>
+
+# Containers
+
+Types:
+
+- <code><a href="./src/resources/containers/containers.ts">ContainerCreateResponse</a></code>
+- <code><a href="./src/resources/containers/containers.ts">ContainerRetrieveResponse</a></code>
+- <code><a href="./src/resources/containers/containers.ts">ContainerListResponse</a></code>
+
+Methods:
+
+- <code title="post /containers">client.containers.<a href="./src/resources/containers/containers.ts">create</a>({ ...params }) -> ContainerCreateResponse</code>
+- <code title="get /containers/{container_id}">client.containers.<a href="./src/resources/containers/containers.ts">retrieve</a>(containerId) -> ContainerRetrieveResponse</code>
+- <code title="get /containers">client.containers.<a href="./src/resources/containers/containers.ts">list</a>({ ...params }) -> ContainerListResponsesPage</code>
+- <code title="delete /containers/{container_id}">client.containers.<a href="./src/resources/containers/containers.ts">del</a>(containerId) -> void</code>
+
+## Files
+
+Types:
+
+- <code><a href="./src/resources/containers/files/files.ts">FileCreateResponse</a></code>
+- <code><a href="./src/resources/containers/files/files.ts">FileRetrieveResponse</a></code>
+- <code><a href="./src/resources/containers/files/files.ts">FileListResponse</a></code>
+
+Methods:
+
+- <code title="post /containers/{container_id}/files">client.containers.files.<a href="./src/resources/containers/files/files.ts">create</a>(containerId, { ...params }) -> FileCreateResponse</code>
+- <code title="get /containers/{container_id}/files/{file_id}">client.containers.files.<a href="./src/resources/containers/files/files.ts">retrieve</a>(containerId, fileId) -> FileRetrieveResponse</code>
+- <code title="get /containers/{container_id}/files">client.containers.files.<a href="./src/resources/containers/files/files.ts">list</a>(containerId, { ...params }) -> FileListResponsesPage</code>
+- <code title="delete /containers/{container_id}/files/{file_id}">client.containers.files.<a href="./src/resources/containers/files/files.ts">del</a>(containerId, fileId) -> void</code>
+
+### Content
+
+Methods:
+
+- <code title="get /containers/{container_id}/files/{file_id}/content">client.containers.files.content.<a href="./src/resources/containers/files/content.ts">retrieve</a>(containerId, fileId) -> void</code>
diff --git examples/azure/realtime/websocket.ts examples/azure/realtime/websocket.ts
index bec74e654..91fe3b7b9 100644
--- examples/azure/realtime/websocket.ts
+++ examples/azure/realtime/websocket.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/azure/realtime/ws.ts examples/azure/realtime/ws.ts
index 6ab7b742a..8b22aeef0 100644
--- examples/azure/realtime/ws.ts
+++ examples/azure/realtime/ws.ts
@@ -40,7 +40,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/websocket.ts examples/realtime/websocket.ts
index 0da131bc3..6fb4740af 100644
--- examples/realtime/websocket.ts
+++ examples/realtime/websocket.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/realtime/ws.ts examples/realtime/ws.ts
index 08c6fbcb6..6cc950b76 100644
--- examples/realtime/ws.ts
+++ examples/realtime/ws.ts
@@ -28,7 +28,7 @@ async function main() {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
diff --git examples/tsconfig.json examples/tsconfig.json
index 6c3477462..3c43903cf 100644
--- examples/tsconfig.json
+++ examples/tsconfig.json
@@ -1,3 +1,3 @@
{
- "extends": "../tsconfig.json"
+ "extends": "../tsconfig.json"
}
diff --git jsr.json jsr.json
index 8eca06e74..a48e4e9ca 100644
--- jsr.json
+++ jsr.json
@@ -1,6 +1,6 @@
{
"name": "@openai/openai",
- "version": "4.96.2",
+ "version": "4.102.0",
"exports": {
".": "./index.ts",
"./helpers/zod": "./helpers/zod.ts",
diff --git package.json package.json
index d563394c0..c1886271b 100644
--- package.json
+++ package.json
@@ -1,6 +1,6 @@
{
"name": "openai",
- "version": "4.96.2",
+ "version": "4.102.0",
"description": "The official TypeScript library for the OpenAI API",
"author": "OpenAI <[email protected]>",
"types": "dist/index.d.ts",
diff --git realtime.md realtime.md
index 2fcd17e9e..7e8d84a3c 100644
--- realtime.md
+++ realtime.md
@@ -39,7 +39,7 @@ rt.socket.on('open', () => {
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
@@ -80,8 +80,7 @@ It is **highly recommended** that you register an `error` event listener and han
const rt = new OpenAIRealtimeWS({ model: 'gpt-4o-realtime-preview-2024-12-17' });
rt.on('error', (err) => {
// in a real world scenario this should be logged somewhere as you
- // likely want to continue procesing events regardless of any errors
+ // likely want to continue processing events regardless of any errors
throw err;
});
-```
-
+```
\ No newline at end of file
diff --git src/index.ts src/index.ts
index 9e8d7ce37..3b4b9a574 100644
--- src/index.ts
+++ src/index.ts
@@ -66,24 +66,31 @@ import {
import { Audio, AudioModel, AudioResponseFormat } from './resources/audio/audio';
import { Beta } from './resources/beta/beta';
import { Chat } from './resources/chat/chat';
+import {
+ ContainerCreateParams,
+ ContainerCreateResponse,
+ ContainerListParams,
+ ContainerListResponse,
+ ContainerListResponsesPage,
+ ContainerRetrieveResponse,
+ Containers,
+} from './resources/containers/containers';
import {
EvalCreateParams,
EvalCreateResponse,
EvalCustomDataSourceConfig,
EvalDeleteResponse,
- EvalLabelModelGrader,
EvalListParams,
EvalListResponse,
EvalListResponsesPage,
EvalRetrieveResponse,
EvalStoredCompletionsDataSourceConfig,
- EvalStringCheckGrader,
- EvalTextSimilarityGrader,
EvalUpdateParams,
EvalUpdateResponse,
Evals,
} from './resources/evals/evals';
import { FineTuning } from './resources/fine-tuning/fine-tuning';
+import { Graders } from './resources/graders/graders';
import { Responses } from './resources/responses/responses';
import {
Upload,
@@ -305,12 +312,14 @@ export class OpenAI extends Core.APIClient {
moderations: API.Moderations = new API.Moderations(this);
models: API.Models = new API.Models(this);
fineTuning: API.FineTuning = new API.FineTuning(this);
+ graders: API.Graders = new API.Graders(this);
vectorStores: API.VectorStores = new API.VectorStores(this);
beta: API.Beta = new API.Beta(this);
batches: API.Batches = new API.Batches(this);
uploads: API.Uploads = new API.Uploads(this);
responses: API.Responses = new API.Responses(this);
evals: API.Evals = new API.Evals(this);
+ containers: API.Containers = new API.Containers(this);
protected override defaultQuery(): Core.DefaultQuery | undefined {
return this._options.defaultQuery;
@@ -366,6 +375,7 @@ OpenAI.Moderations = Moderations;
OpenAI.Models = Models;
OpenAI.ModelsPage = ModelsPage;
OpenAI.FineTuning = FineTuning;
+OpenAI.Graders = Graders;
OpenAI.VectorStores = VectorStores;
OpenAI.VectorStoresPage = VectorStoresPage;
OpenAI.VectorStoreSearchResponsesPage = VectorStoreSearchResponsesPage;
@@ -376,6 +386,8 @@ OpenAI.Uploads = UploadsAPIUploads;
OpenAI.Responses = Responses;
OpenAI.Evals = Evals;
OpenAI.EvalListResponsesPage = EvalListResponsesPage;
+OpenAI.Containers = Containers;
+OpenAI.ContainerListResponsesPage = ContainerListResponsesPage;
export declare namespace OpenAI {
export type RequestOptions = Core.RequestOptions;
@@ -487,6 +499,8 @@ export declare namespace OpenAI {
export { FineTuning as FineTuning };
+ export { Graders as Graders };
+
export {
VectorStores as VectorStores,
type AutoFileChunkingStrategyParam as AutoFileChunkingStrategyParam,
@@ -531,10 +545,7 @@ export declare namespace OpenAI {
export {
Evals as Evals,
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
@@ -546,6 +557,16 @@ export declare namespace OpenAI {
type EvalListParams as EvalListParams,
};
+ export {
+ Containers as Containers,
+ type ContainerCreateResponse as ContainerCreateResponse,
+ type ContainerRetrieveResponse as ContainerRetrieveResponse,
+ type ContainerListResponse as ContainerListResponse,
+ ContainerListResponsesPage as ContainerListResponsesPage,
+ type ContainerCreateParams as ContainerCreateParams,
+ type ContainerListParams as ContainerListParams,
+ };
+
export type AllModels = API.AllModels;
export type ChatModel = API.ChatModel;
export type ComparisonFilter = API.ComparisonFilter;
diff --git src/resources/audio/speech.ts src/resources/audio/speech.ts
index 4b99ee5f4..ccd37c092 100644
--- src/resources/audio/speech.ts
+++ src/resources/audio/speech.ts
@@ -7,6 +7,18 @@ import { type Response } from '../../_shims/index';
export class Speech extends APIResource {
/**
* Generates audio from the input text.
+ *
+ * @example
+ * ```ts
+ * const speech = await client.audio.speech.create({
+ * input: 'input',
+ * model: 'string',
+ * voice: 'ash',
+ * });
+ *
+ * const content = await speech.blob();
+ * console.log(content);
+ * ```
*/
create(body: SpeechCreateParams, options?: Core.RequestOptions): Core.APIPromise<Response> {
return this._client.post('/audio/speech', {
@@ -66,7 +78,7 @@ export interface SpeechCreateParams {
/**
* The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is
- * the default.
+ * the default. Does not work with `gpt-4o-mini-tts`.
*/
speed?: number;
}
diff --git src/resources/audio/transcriptions.ts src/resources/audio/transcriptions.ts
index ba4fec6c5..9e5310874 100644
--- src/resources/audio/transcriptions.ts
+++ src/resources/audio/transcriptions.ts
@@ -9,6 +9,15 @@ import { Stream } from '../../streaming';
export class Transcriptions extends APIResource {
/**
* Transcribes audio into the input language.
+ *
+ * @example
+ * ```ts
+ * const transcription =
+ * await client.audio.transcriptions.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'gpt-4o-transcribe',
+ * });
+ * ```
*/
create(
body: TranscriptionCreateParamsNonStreaming<'json' | undefined>,
@@ -315,6 +324,14 @@ export interface TranscriptionCreateParamsBase<
*/
model: (string & {}) | AudioAPI.AudioModel;
+ /**
+ * Controls how the audio is cut into chunks. When set to `"auto"`, the server
+ * first normalizes loudness and then uses voice activity detection (VAD) to choose
+ * boundaries. `server_vad` object can be provided to tweak VAD detection
+ * parameters manually. If unset, the audio is transcribed as a single block.
+ */
+ chunking_strategy?: 'auto' | TranscriptionCreateParams.VadConfig | null;
+
/**
* Additional information to include in the transcription response. `logprobs` will
* return the log probabilities of the tokens in the response to understand the
@@ -378,6 +395,32 @@ export interface TranscriptionCreateParamsBase<
}
export namespace TranscriptionCreateParams {
+ export interface VadConfig {
+ /**
+ * Must be set to `server_vad` to enable manual chunking using server side VAD.
+ */
+ type: 'server_vad';
+
+ /**
+ * Amount of audio to include before the VAD detected speech (in milliseconds).
+ */
+ prefix_padding_ms?: number;
+
+ /**
+ * Duration of silence to detect speech stop (in milliseconds). With shorter values
+ * the model will respond more quickly, but may jump in on short pauses from the
+ * user.
+ */
+ silence_duration_ms?: number;
+
+ /**
+ * Sensitivity threshold (0.0 to 1.0) for voice activity detection. A higher
+ * threshold will require louder audio to activate the model, and thus might
+ * perform better in noisy environments.
+ */
+ threshold?: number;
+ }
+
export type TranscriptionCreateParamsNonStreaming = TranscriptionsAPI.TranscriptionCreateParamsNonStreaming;
export type TranscriptionCreateParamsStreaming = TranscriptionsAPI.TranscriptionCreateParamsStreaming;
}
diff --git src/resources/audio/translations.ts src/resources/audio/translations.ts
index df312f876..1edb71a7d 100644
--- src/resources/audio/translations.ts
+++ src/resources/audio/translations.ts
@@ -8,6 +8,14 @@ import * as TranscriptionsAPI from './transcriptions';
export class Translations extends APIResource {
/**
* Translates audio into English.
+ *
+ * @example
+ * ```ts
+ * const translation = await client.audio.translations.create({
+ * file: fs.createReadStream('speech.mp3'),
+ * model: 'whisper-1',
+ * });
+ * ```
*/
create(
body: TranslationCreateParams<'json' | undefined>,
diff --git src/resources/beta/assistants.ts src/resources/beta/assistants.ts
index 00a6ff2cf..95581bbc8 100644
--- src/resources/beta/assistants.ts
+++ src/resources/beta/assistants.ts
@@ -14,6 +14,13 @@ import { AssistantStream } from '../../lib/AssistantStream';
export class Assistants extends APIResource {
/**
* Create an assistant with a model and instructions.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.create({
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: AssistantCreateParams, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.post('/assistants', {
@@ -25,6 +32,13 @@ export class Assistants extends APIResource {
/**
* Retrieves an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.retrieve(
+ * 'assistant_id',
+ * );
+ * ```
*/
retrieve(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<Assistant> {
return this._client.get(`/assistants/${assistantId}`, {
@@ -35,6 +49,13 @@ export class Assistants extends APIResource {
/**
* Modifies an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistant = await client.beta.assistants.update(
+ * 'assistant_id',
+ * );
+ * ```
*/
update(
assistantId: string,
@@ -50,6 +71,14 @@ export class Assistants extends APIResource {
/**
* Returns a list of assistants.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const assistant of client.beta.assistants.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: AssistantListParams,
@@ -72,6 +101,13 @@ export class Assistants extends APIResource {
/**
* Delete an assistant.
+ *
+ * @example
+ * ```ts
+ * const assistantDeleted = await client.beta.assistants.del(
+ * 'assistant_id',
+ * );
+ * ```
*/
del(assistantId: string, options?: Core.RequestOptions): Core.APIPromise<AssistantDeleted> {
return this._client.delete(`/assistants/${assistantId}`, {
diff --git src/resources/beta/realtime/realtime.ts src/resources/beta/realtime/realtime.ts
index 5012b1edd..26fba883e 100644
--- src/resources/beta/realtime/realtime.ts
+++ src/resources/beta/realtime/realtime.ts
@@ -2571,7 +2571,7 @@ export interface TranscriptionSessionUpdatedEvent {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
session: TranscriptionSessionsAPI.TranscriptionSession;
diff --git src/resources/beta/realtime/sessions.ts src/resources/beta/realtime/sessions.ts
index 28a44431e..a55a2678c 100644
--- src/resources/beta/realtime/sessions.ts
+++ src/resources/beta/realtime/sessions.ts
@@ -12,6 +12,12 @@ export class Sessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const session =
+ * await client.beta.realtime.sessions.create();
+ * ```
*/
create(body: SessionCreateParams, options?: Core.RequestOptions): Core.APIPromise<SessionCreateResponse> {
return this._client.post('/realtime/sessions', {
diff --git src/resources/beta/realtime/transcription-sessions.ts src/resources/beta/realtime/transcription-sessions.ts
index a54ec1125..83e8c47ad 100644
--- src/resources/beta/realtime/transcription-sessions.ts
+++ src/resources/beta/realtime/transcription-sessions.ts
@@ -12,6 +12,12 @@ export class TranscriptionSessions extends APIResource {
* It responds with a session object, plus a `client_secret` key which contains a
* usable ephemeral API token that can be used to authenticate browser clients for
* the Realtime API.
+ *
+ * @example
+ * ```ts
+ * const transcriptionSession =
+ * await client.beta.realtime.transcriptionSessions.create();
+ * ```
*/
create(
body: TranscriptionSessionCreateParams,
@@ -29,7 +35,7 @@ export class TranscriptionSessions extends APIResource {
* A new Realtime transcription session configuration.
*
* When a session is created on the server via REST API, the session object also
- * contains an ephemeral key. Default TTL for keys is one minute. This property is
+ * contains an ephemeral key. Default TTL for keys is 10 minutes. This property is
* not present when a session is updated via the WebSocket API.
*/
export interface TranscriptionSession {
diff --git src/resources/beta/threads/messages.ts src/resources/beta/threads/messages.ts
index 29fd2b29f..c3834ebe6 100644
--- src/resources/beta/threads/messages.ts
+++ src/resources/beta/threads/messages.ts
@@ -10,6 +10,14 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Messages extends APIResource {
/**
* Create a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.create(
+ * 'thread_id',
+ * { content: 'string', role: 'user' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -25,6 +33,14 @@ export class Messages extends APIResource {
/**
* Retrieve a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.retrieve(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
retrieve(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<Message> {
return this._client.get(`/threads/${threadId}/messages/${messageId}`, {
@@ -35,6 +51,14 @@ export class Messages extends APIResource {
/**
* Modifies a message.
+ *
+ * @example
+ * ```ts
+ * const message = await client.beta.threads.messages.update(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -51,6 +75,16 @@ export class Messages extends APIResource {
/**
* Returns a list of messages for a given thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const message of client.beta.threads.messages.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -75,6 +109,15 @@ export class Messages extends APIResource {
/**
* Deletes a message.
+ *
+ * @example
+ * ```ts
+ * const messageDeleted =
+ * await client.beta.threads.messages.del(
+ * 'thread_id',
+ * 'message_id',
+ * );
+ * ```
*/
del(threadId: string, messageId: string, options?: Core.RequestOptions): Core.APIPromise<MessageDeleted> {
return this._client.delete(`/threads/${threadId}/messages/${messageId}`, {
diff --git src/resources/beta/threads/runs/runs.ts src/resources/beta/threads/runs/runs.ts
index 15bfb4204..25356df3c 100644
--- src/resources/beta/threads/runs/runs.ts
+++ src/resources/beta/threads/runs/runs.ts
@@ -45,6 +45,14 @@ export class Runs extends APIResource {
/**
* Create a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.create(
+ * 'thread_id',
+ * { assistant_id: 'assistant_id' },
+ * );
+ * ```
*/
create(
threadId: string,
@@ -78,6 +86,14 @@ export class Runs extends APIResource {
/**
* Retrieves a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
retrieve(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.get(`/threads/${threadId}/runs/${runId}`, {
@@ -88,6 +104,14 @@ export class Runs extends APIResource {
/**
* Modifies a run.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.update(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
update(
threadId: string,
@@ -104,6 +128,16 @@ export class Runs extends APIResource {
/**
* Returns a list of runs belonging to a thread.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const run of client.beta.threads.runs.list(
+ * 'thread_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
@@ -128,6 +162,14 @@ export class Runs extends APIResource {
/**
* Cancels a run that is `in_progress`.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.runs.cancel(
+ * 'thread_id',
+ * 'run_id',
+ * );
+ * ```
*/
cancel(threadId: string, runId: string, options?: Core.RequestOptions): Core.APIPromise<Run> {
return this._client.post(`/threads/${threadId}/runs/${runId}/cancel`, {
@@ -229,6 +271,16 @@ export class Runs extends APIResource {
* `submit_tool_outputs`, this endpoint can be used to submit the outputs from the
* tool calls once they're all completed. All outputs must be submitted in a single
* request.
+ *
+ * @example
+ * ```ts
+ * const run =
+ * await client.beta.threads.runs.submitToolOutputs(
+ * 'thread_id',
+ * 'run_id',
+ * { tool_outputs: [{}] },
+ * );
+ * ```
*/
submitToolOutputs(
threadId: string,
diff --git src/resources/beta/threads/runs/steps.ts src/resources/beta/threads/runs/steps.ts
index c491b4e83..abd8d40ed 100644
--- src/resources/beta/threads/runs/steps.ts
+++ src/resources/beta/threads/runs/steps.ts
@@ -10,6 +10,16 @@ import { CursorPage, type CursorPageParams } from '../../../../pagination';
export class Steps extends APIResource {
/**
* Retrieves a run step.
+ *
+ * @example
+ * ```ts
+ * const runStep =
+ * await client.beta.threads.runs.steps.retrieve(
+ * 'thread_id',
+ * 'run_id',
+ * 'step_id',
+ * );
+ * ```
*/
retrieve(
threadId: string,
@@ -43,6 +53,17 @@ export class Steps extends APIResource {
/**
* Returns a list of run steps belonging to a run.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const runStep of client.beta.threads.runs.steps.list(
+ * 'thread_id',
+ * 'run_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
threadId: string,
diff --git src/resources/beta/threads/threads.ts src/resources/beta/threads/threads.ts
index 2e5ab1cc8..c0c6bc8e4 100644
--- src/resources/beta/threads/threads.ts
+++ src/resources/beta/threads/threads.ts
@@ -73,6 +73,11 @@ export class Threads extends APIResource {
/**
* Create a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.create();
+ * ```
*/
create(body?: ThreadCreateParams, options?: Core.RequestOptions): Core.APIPromise<Thread>;
create(options?: Core.RequestOptions): Core.APIPromise<Thread>;
@@ -92,6 +97,13 @@ export class Threads extends APIResource {
/**
* Retrieves a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.retrieve(
+ * 'thread_id',
+ * );
+ * ```
*/
retrieve(threadId: string, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.get(`/threads/${threadId}`, {
@@ -102,6 +114,13 @@ export class Threads extends APIResource {
/**
* Modifies a thread.
+ *
+ * @example
+ * ```ts
+ * const thread = await client.beta.threads.update(
+ * 'thread_id',
+ * );
+ * ```
*/
update(threadId: string, body: ThreadUpdateParams, options?: Core.RequestOptions): Core.APIPromise<Thread> {
return this._client.post(`/threads/${threadId}`, {
@@ -113,6 +132,13 @@ export class Threads extends APIResource {
/**
* Delete a thread.
+ *
+ * @example
+ * ```ts
+ * const threadDeleted = await client.beta.threads.del(
+ * 'thread_id',
+ * );
+ * ```
*/
del(threadId: string, options?: Core.RequestOptions): Core.APIPromise<ThreadDeleted> {
return this._client.delete(`/threads/${threadId}`, {
@@ -123,6 +149,13 @@ export class Threads extends APIResource {
/**
* Create a thread and run it in one request.
+ *
+ * @example
+ * ```ts
+ * const run = await client.beta.threads.createAndRun({
+ * assistant_id: 'assistant_id',
+ * });
+ * ```
*/
createAndRun(
body: ThreadCreateAndRunParamsNonStreaming,
diff --git src/resources/chat/completions/completions.ts src/resources/chat/completions/completions.ts
index 17edac02c..6481f8e0f 100644
--- src/resources/chat/completions/completions.ts
+++ src/resources/chat/completions/completions.ts
@@ -33,6 +33,16 @@ export class Completions extends APIResource {
* supported for reasoning models are noted below. For the current state of
* unsupported parameters in reasoning models,
* [refer to the reasoning guide](https://platform.openai.com/docs/guides/reasoning).
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.create(
+ * {
+ * messages: [{ content: 'string', role: 'developer' }],
+ * model: 'gpt-4o',
+ * },
+ * );
+ * ```
*/
create(
body: ChatCompletionCreateParamsNonStreaming,
@@ -58,6 +68,12 @@ export class Completions extends APIResource {
/**
* Get a stored chat completion. Only Chat Completions that have been created with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion =
+ * await client.chat.completions.retrieve('completion_id');
+ * ```
*/
retrieve(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletion> {
return this._client.get(`/chat/completions/${completionId}`, options);
@@ -67,6 +83,14 @@ export class Completions extends APIResource {
* Modify a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be modified. Currently, the only
* supported modification is to update the `metadata` field.
+ *
+ * @example
+ * ```ts
+ * const chatCompletion = await client.chat.completions.update(
+ * 'completion_id',
+ * { metadata: { foo: 'string' } },
+ * );
+ * ```
*/
update(
completionId: string,
@@ -79,6 +103,14 @@ export class Completions extends APIResource {
/**
* List stored Chat Completions. Only Chat Completions that have been stored with
* the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletion of client.chat.completions.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: ChatCompletionListParams,
@@ -98,6 +130,12 @@ export class Completions extends APIResource {
/**
* Delete a stored chat completion. Only Chat Completions that have been created
* with the `store` parameter set to `true` can be deleted.
+ *
+ * @example
+ * ```ts
+ * const chatCompletionDeleted =
+ * await client.chat.completions.del('completion_id');
+ * ```
*/
del(completionId: string, options?: Core.RequestOptions): Core.APIPromise<ChatCompletionDeleted> {
return this._client.delete(`/chat/completions/${completionId}`, options);
@@ -1169,7 +1207,7 @@ export interface ChatCompletionCreateParamsBase {
frequency_penalty?: number | null;
/**
- * Deprecated in favor of `tool_choice`.
+ * @deprecated Deprecated in favor of `tool_choice`.
*
* Controls which (if any) function is called by the model.
*
@@ -1187,7 +1225,7 @@ export interface ChatCompletionCreateParamsBase {
function_call?: 'none' | 'auto' | ChatCompletionFunctionCallOption;
/**
- * Deprecated in favor of `tools`.
+ * @deprecated Deprecated in favor of `tools`.
*
* A list of functions the model may generate JSON inputs for.
*/
@@ -1220,8 +1258,8 @@ export interface ChatCompletionCreateParamsBase {
max_completion_tokens?: number | null;
/**
- * The maximum number of [tokens](/tokenizer) that can be generated in the chat
- * completion. This value can be used to control
+ * @deprecated The maximum number of [tokens](/tokenizer) that can be generated in
+ * the chat completion. This value can be used to control
* [costs](https://openai.com/api/pricing/) for text generated via API.
*
* This value is now deprecated in favor of `max_completion_tokens`, and is not
diff --git src/resources/chat/completions/messages.ts src/resources/chat/completions/messages.ts
index 519a33aff..ab3eb73f6 100644
--- src/resources/chat/completions/messages.ts
+++ src/resources/chat/completions/messages.ts
@@ -11,6 +11,16 @@ export class Messages extends APIResource {
/**
* Get the messages in a stored chat completion. Only Chat Completions that have
* been created with the `store` parameter set to `true` will be returned.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const chatCompletionStoreMessage of client.chat.completions.messages.list(
+ * 'completion_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
completionId: string,
diff --git src/resources/completions.ts src/resources/completions.ts
index 5cbec5e3c..07cb49ed9 100644
--- src/resources/completions.ts
+++ src/resources/completions.ts
@@ -10,6 +10,14 @@ import { Stream } from '../streaming';
export class Completions extends APIResource {
/**
* Creates a completion for the provided prompt and parameters.
+ *
+ * @example
+ * ```ts
+ * const completion = await client.completions.create({
+ * model: 'string',
+ * prompt: 'This is a test.',
+ * });
+ * ```
*/
create(body: CompletionCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Completion>;
create(
diff --git a/src/resources/containers.ts b/src/resources/containers.ts
new file mode 100644
index 000000000..c8f3a6456
--- /dev/null
+++ src/resources/containers.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './containers/index';
diff --git a/src/resources/containers/containers.ts b/src/resources/containers/containers.ts
new file mode 100644
index 000000000..66303dc61
--- /dev/null
+++ src/resources/containers/containers.ts
@@ -0,0 +1,291 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import { isRequestOptions } from '../../core';
+import * as Core from '../../core';
+import * as FilesAPI from './files/files';
+import {
+ FileCreateParams,
+ FileCreateResponse,
+ FileListParams,
+ FileListResponse,
+ FileListResponsesPage,
+ FileRetrieveResponse,
+ Files,
+} from './files/files';
+import { CursorPage, type CursorPageParams } from '../../pagination';
+
+export class Containers extends APIResource {
+ files: FilesAPI.Files = new FilesAPI.Files(this._client);
+
+ /**
+ * Create Container
+ */
+ create(
+ body: ContainerCreateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<ContainerCreateResponse> {
+ return this._client.post('/containers', { body, ...options });
+ }
+
+ /**
+ * Retrieve Container
+ */
+ retrieve(containerId: string, options?: Core.RequestOptions): Core.APIPromise<ContainerRetrieveResponse> {
+ return this._client.get(`/containers/${containerId}`, options);
+ }
+
+ /**
+ * List Containers
+ */
+ list(
+ query?: ContainerListParams,
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<ContainerListResponsesPage, ContainerListResponse>;
+ list(options?: Core.RequestOptions): Core.PagePromise<ContainerListResponsesPage, ContainerListResponse>;
+ list(
+ query: ContainerListParams | Core.RequestOptions = {},
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<ContainerListResponsesPage, ContainerListResponse> {
+ if (isRequestOptions(query)) {
+ return this.list({}, query);
+ }
+ return this._client.getAPIList('/containers', ContainerListResponsesPage, { query, ...options });
+ }
+
+ /**
+ * Delete Container
+ */
+ del(containerId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.delete(`/containers/${containerId}`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
+}
+
+export class ContainerListResponsesPage extends CursorPage<ContainerListResponse> {}
+
+export interface ContainerCreateResponse {
+ /**
+ * Unique identifier for the container.
+ */
+ id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the container was created.
+ */
+ created_at: number;
+
+ /**
+ * Name of the container.
+ */
+ name: string;
+
+ /**
+ * The type of this object.
+ */
+ object: string;
+
+ /**
+ * Status of the container (e.g., active, deleted).
+ */
+ status: string;
+
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ expires_after?: ContainerCreateResponse.ExpiresAfter;
+}
+
+export namespace ContainerCreateResponse {
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ export interface ExpiresAfter {
+ /**
+ * The reference point for the expiration.
+ */
+ anchor?: 'last_active_at';
+
+ /**
+ * The number of minutes after the anchor before the container expires.
+ */
+ minutes?: number;
+ }
+}
+
+export interface ContainerRetrieveResponse {
+ /**
+ * Unique identifier for the container.
+ */
+ id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the container was created.
+ */
+ created_at: number;
+
+ /**
+ * Name of the container.
+ */
+ name: string;
+
+ /**
+ * The type of this object.
+ */
+ object: string;
+
+ /**
+ * Status of the container (e.g., active, deleted).
+ */
+ status: string;
+
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ expires_after?: ContainerRetrieveResponse.ExpiresAfter;
+}
+
+export namespace ContainerRetrieveResponse {
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ export interface ExpiresAfter {
+ /**
+ * The reference point for the expiration.
+ */
+ anchor?: 'last_active_at';
+
+ /**
+ * The number of minutes after the anchor before the container expires.
+ */
+ minutes?: number;
+ }
+}
+
+export interface ContainerListResponse {
+ /**
+ * Unique identifier for the container.
+ */
+ id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the container was created.
+ */
+ created_at: number;
+
+ /**
+ * Name of the container.
+ */
+ name: string;
+
+ /**
+ * The type of this object.
+ */
+ object: string;
+
+ /**
+ * Status of the container (e.g., active, deleted).
+ */
+ status: string;
+
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ expires_after?: ContainerListResponse.ExpiresAfter;
+}
+
+export namespace ContainerListResponse {
+ /**
+ * The container will expire after this time period. The anchor is the reference
+ * point for the expiration. The minutes is the number of minutes after the anchor
+ * before the container expires.
+ */
+ export interface ExpiresAfter {
+ /**
+ * The reference point for the expiration.
+ */
+ anchor?: 'last_active_at';
+
+ /**
+ * The number of minutes after the anchor before the container expires.
+ */
+ minutes?: number;
+ }
+}
+
+export interface ContainerCreateParams {
+ /**
+ * Name of the container to create.
+ */
+ name: string;
+
+ /**
+ * Container expiration time in seconds relative to the 'anchor' time.
+ */
+ expires_after?: ContainerCreateParams.ExpiresAfter;
+
+ /**
+ * IDs of files to copy to the container.
+ */
+ file_ids?: Array<string>;
+}
+
+export namespace ContainerCreateParams {
+ /**
+ * Container expiration time in seconds relative to the 'anchor' time.
+ */
+ export interface ExpiresAfter {
+ /**
+ * Time anchor for the expiration time. Currently only 'last_active_at' is
+ * supported.
+ */
+ anchor: 'last_active_at';
+
+ minutes: number;
+ }
+}
+
+export interface ContainerListParams extends CursorPageParams {
+ /**
+ * Sort order by the `created_at` timestamp of the objects. `asc` for ascending
+ * order and `desc` for descending order.
+ */
+ order?: 'asc' | 'desc';
+}
+
+Containers.ContainerListResponsesPage = ContainerListResponsesPage;
+Containers.Files = Files;
+Containers.FileListResponsesPage = FileListResponsesPage;
+
+export declare namespace Containers {
+ export {
+ type ContainerCreateResponse as ContainerCreateResponse,
+ type ContainerRetrieveResponse as ContainerRetrieveResponse,
+ type ContainerListResponse as ContainerListResponse,
+ ContainerListResponsesPage as ContainerListResponsesPage,
+ type ContainerCreateParams as ContainerCreateParams,
+ type ContainerListParams as ContainerListParams,
+ };
+
+ export {
+ Files as Files,
+ type FileCreateResponse as FileCreateResponse,
+ type FileRetrieveResponse as FileRetrieveResponse,
+ type FileListResponse as FileListResponse,
+ FileListResponsesPage as FileListResponsesPage,
+ type FileCreateParams as FileCreateParams,
+ type FileListParams as FileListParams,
+ };
+}
diff --git a/src/resources/containers/files.ts b/src/resources/containers/files.ts
new file mode 100644
index 000000000..46a5299c1
--- /dev/null
+++ src/resources/containers/files.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './files/index';
diff --git a/src/resources/containers/files/content.ts b/src/resources/containers/files/content.ts
new file mode 100644
index 000000000..efb33a87f
--- /dev/null
+++ src/resources/containers/files/content.ts
@@ -0,0 +1,16 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+
+export class Content extends APIResource {
+ /**
+ * Retrieve Container File Content
+ */
+ retrieve(containerId: string, fileId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.get(`/containers/${containerId}/files/${fileId}/content`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
+}
diff --git a/src/resources/containers/files/files.ts b/src/resources/containers/files/files.ts
new file mode 100644
index 000000000..0c249df0f
--- /dev/null
+++ src/resources/containers/files/files.ts
@@ -0,0 +1,222 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import { isRequestOptions } from '../../../core';
+import * as Core from '../../../core';
+import * as ContentAPI from './content';
+import { Content } from './content';
+import { CursorPage, type CursorPageParams } from '../../../pagination';
+
+export class Files extends APIResource {
+ content: ContentAPI.Content = new ContentAPI.Content(this._client);
+
+ /**
+ * Create a Container File
+ *
+ * You can send either a multipart/form-data request with the raw file content, or
+ * a JSON request with a file ID.
+ */
+ create(
+ containerId: string,
+ body: FileCreateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<FileCreateResponse> {
+ return this._client.post(`/containers/${containerId}/files`, { body, ...options });
+ }
+
+ /**
+ * Retrieve Container File
+ */
+ retrieve(
+ containerId: string,
+ fileId: string,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<FileRetrieveResponse> {
+ return this._client.get(`/containers/${containerId}/files/${fileId}`, options);
+ }
+
+ /**
+ * List Container files
+ */
+ list(
+ containerId: string,
+ query?: FileListParams,
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<FileListResponsesPage, FileListResponse>;
+ list(
+ containerId: string,
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<FileListResponsesPage, FileListResponse>;
+ list(
+ containerId: string,
+ query: FileListParams | Core.RequestOptions = {},
+ options?: Core.RequestOptions,
+ ): Core.PagePromise<FileListResponsesPage, FileListResponse> {
+ if (isRequestOptions(query)) {
+ return this.list(containerId, {}, query);
+ }
+ return this._client.getAPIList(`/containers/${containerId}/files`, FileListResponsesPage, {
+ query,
+ ...options,
+ });
+ }
+
+ /**
+ * Delete Container File
+ */
+ del(containerId: string, fileId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.delete(`/containers/${containerId}/files/${fileId}`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
+}
+
+export class FileListResponsesPage extends CursorPage<FileListResponse> {}
+
+export interface FileCreateResponse {
+ /**
+ * Unique identifier for the file.
+ */
+ id: string;
+
+ /**
+ * Size of the file in bytes.
+ */
+ bytes: number;
+
+ /**
+ * The container this file belongs to.
+ */
+ container_id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the file was created.
+ */
+ created_at: number;
+
+ /**
+ * The type of this object (`container.file`).
+ */
+ object: 'container.file';
+
+ /**
+ * Path of the file in the container.
+ */
+ path: string;
+
+ /**
+ * Source of the file (e.g., `user`, `assistant`).
+ */
+ source: string;
+}
+
+export interface FileRetrieveResponse {
+ /**
+ * Unique identifier for the file.
+ */
+ id: string;
+
+ /**
+ * Size of the file in bytes.
+ */
+ bytes: number;
+
+ /**
+ * The container this file belongs to.
+ */
+ container_id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the file was created.
+ */
+ created_at: number;
+
+ /**
+ * The type of this object (`container.file`).
+ */
+ object: 'container.file';
+
+ /**
+ * Path of the file in the container.
+ */
+ path: string;
+
+ /**
+ * Source of the file (e.g., `user`, `assistant`).
+ */
+ source: string;
+}
+
+export interface FileListResponse {
+ /**
+ * Unique identifier for the file.
+ */
+ id: string;
+
+ /**
+ * Size of the file in bytes.
+ */
+ bytes: number;
+
+ /**
+ * The container this file belongs to.
+ */
+ container_id: string;
+
+ /**
+ * Unix timestamp (in seconds) when the file was created.
+ */
+ created_at: number;
+
+ /**
+ * The type of this object (`container.file`).
+ */
+ object: 'container.file';
+
+ /**
+ * Path of the file in the container.
+ */
+ path: string;
+
+ /**
+ * Source of the file (e.g., `user`, `assistant`).
+ */
+ source: string;
+}
+
+export interface FileCreateParams {
+ /**
+ * The File object (not file name) to be uploaded.
+ */
+ file?: Core.Uploadable;
+
+ /**
+ * Name of the file to create.
+ */
+ file_id?: string;
+}
+
+export interface FileListParams extends CursorPageParams {
+ /**
+ * Sort order by the `created_at` timestamp of the objects. `asc` for ascending
+ * order and `desc` for descending order.
+ */
+ order?: 'asc' | 'desc';
+}
+
+Files.FileListResponsesPage = FileListResponsesPage;
+Files.Content = Content;
+
+export declare namespace Files {
+ export {
+ type FileCreateResponse as FileCreateResponse,
+ type FileRetrieveResponse as FileRetrieveResponse,
+ type FileListResponse as FileListResponse,
+ FileListResponsesPage as FileListResponsesPage,
+ type FileCreateParams as FileCreateParams,
+ type FileListParams as FileListParams,
+ };
+
+ export { Content as Content };
+}
diff --git a/src/resources/containers/files/index.ts b/src/resources/containers/files/index.ts
new file mode 100644
index 000000000..7305ab706
--- /dev/null
+++ src/resources/containers/files/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Content } from './content';
+export {
+ FileListResponsesPage,
+ Files,
+ type FileCreateResponse,
+ type FileRetrieveResponse,
+ type FileListResponse,
+ type FileCreateParams,
+ type FileListParams,
+} from './files';
diff --git a/src/resources/containers/index.ts b/src/resources/containers/index.ts
new file mode 100644
index 000000000..91145bbe0
--- /dev/null
+++ src/resources/containers/index.ts
@@ -0,0 +1,20 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ ContainerListResponsesPage,
+ Containers,
+ type ContainerCreateResponse,
+ type ContainerRetrieveResponse,
+ type ContainerListResponse,
+ type ContainerCreateParams,
+ type ContainerListParams,
+} from './containers';
+export {
+ FileListResponsesPage,
+ Files,
+ type FileCreateResponse,
+ type FileRetrieveResponse,
+ type FileListResponse,
+ type FileCreateParams,
+ type FileListParams,
+} from './files/index';
diff --git src/resources/embeddings.ts src/resources/embeddings.ts
index a4be9ca3c..fb02a7654 100644
--- src/resources/embeddings.ts
+++ src/resources/embeddings.ts
@@ -6,6 +6,15 @@ import * as Core from '../core';
export class Embeddings extends APIResource {
/**
* Creates an embedding vector representing the input text.
+ *
+ * @example
+ * ```ts
+ * const createEmbeddingResponse =
+ * await client.embeddings.create({
+ * input: 'The quick brown fox jumped over the lazy dog',
+ * model: 'text-embedding-3-small',
+ * });
+ * ```
*/
create(
body: EmbeddingCreat,eParams,
@@ -121,11 +130,12 @@ export interface EmbeddingCreateParams {
* Input text to embed, encoded as a string or array of tokens. To embed multiple
* inputs in a single request, pass an array of strings or array of token arrays.
* The input must not exceed the max input tokens for the model (8192 tokens for
- * `text-embedding-ada-002`), cannot be an empty string, and any array must be 2048
+ * all embedding models), cannot be an empty string, and any array must be 2048
* dimensions or less.
* [Example Python code](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)
- * for counting tokens. Some models may also impose a limit on total number of
- * tokens summed across inputs.
+ * for counting tokens. In addition to the per-input token limit, all embedding
+ * models enforce a maximum of 300,000 tokens summed across all inputs in a single
+ * request.
*/
input: string | Array<string> | Array<number> | Array<Array<number>>;
diff --git src/resources/evals/evals.ts src/resources/evals/evals.ts
index caef7acc0..05a656619 100644
--- src/resources/evals/evals.ts
+++ src/resources/evals/evals.ts
@@ -4,6 +4,7 @@ import { APIResource } from '../../resource';
import { isRequestOptions } from '../../core';
import * as Core from '../../core';
import * as Shared from '../shared';
+import * as GraderModelsAPI from '../graders/grader-models';
import * as ResponsesAPI from '../responses/responses';
import * as RunsAPI from './runs/runs';
import {
@@ -27,7 +28,8 @@ export class Evals extends APIResource {
/**
* Create the structure of an evaluation that can be used to test a model's
- * performance. An evaluation is a set of testing criteria and a datasource. After
+ * performance. An evaluation is a set of testing criteria and the config for a
+ * data source, which dictates the schema of the data used in the evaluation. After
* creating an evaluation, you can run it on different models and model parameters.
* We support several types of graders and datasources. For more information, see
* the [Evals guide](https://platform.openai.com/docs/guides/evals).
@@ -104,88 +106,7 @@ export interface EvalCustomDataSourceConfig {
}
/**
- * A LabelModelGrader object which uses a model to assign labels to each item in
- * the evaluation.
- */
-export interface EvalLabelModelGrader {
- input: Array<EvalLabelModelGrader.Input>;
-
- /**
- * The labels to assign to each item in the evaluation.
- */
- labels: Array<string>;
-
- /**
- * The model to use for the evaluation. Must support structured outputs.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The labels that indicate a passing result. Must be a subset of labels.
- */
- passing_labels: Array<string>;
-
- /**
- * The object type, which is always `label_model`.
- */
- type: 'label_model';
-}
-
-export namespace EvalLabelModelGrader {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
-}
-
-/**
- * A StoredCompletionsDataSourceConfig which specifies the metadata property of
- * your stored completions query. This is usually metadata like `usecase=chatbot`
- * or `prompt-version=v2`, etc. The schema returned by this data source config is
- * used to defined what variables are available in your evals. `item` and `sample`
- * are both defined when using this data source config.
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
*/
export interface EvalStoredCompletionsDataSourceConfig {
/**
@@ -210,90 +131,13 @@ export interface EvalStoredCompletionsDataSourceConfig {
metadata?: Shared.Metadata | null;
}
-/**
- * A StringCheckGrader object that performs a string comparison between input and
- * reference using a specified operation.
- */
-export interface EvalStringCheckGrader {
- /**
- * The input text. This may include template strings.
- */
- input: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
- */
- operation: 'eq' | 'ne' | 'like' | 'ilike';
-
- /**
- * The reference text. This may include template strings.
- */
- reference: string;
-
- /**
- * The object type, which is always `string_check`.
- */
- type: 'string_check';
-}
-
-/**
- * A TextSimilarityGrader object which grades text based on similarity metrics.
- */
-export interface EvalTextSimilarityGrader {
- /**
- * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
- * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
- */
- evaluation_metric:
- | 'fuzzy_match'
- | 'bleu'
- | 'gleu'
- | 'meteor'
- | 'rouge_1'
- | 'rouge_2'
- | 'rouge_3'
- | 'rouge_4'
- | 'rouge_5'
- | 'rouge_l';
-
- /**
- * The text being graded.
- */
- input: string;
-
- /**
- * A float score where a value greater than or equal indicates a passing grade.
- */
- pass_threshold: number;
-
- /**
- * The text being graded against.
- */
- reference: string;
-
- /**
- * The type of grader.
- */
- type: 'text_similarity';
-
- /**
- * The name of the grader.
- */
- name?: string;
-}
-
/**
* An Eval object with a data source config and testing criteria. An Eval
* represents a task to be done for your LLM integration. Like:
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalCreateResponse {
/**
@@ -309,7 +153,10 @@ export interface EvalCreateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalCreateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -335,39 +182,59 @@ export interface EvalCreateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalCreateResponse.Python
- | EvalCreateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateResponse.EvalGraderTextSimilarity
+ | EvalCreateResponse.EvalGraderPython
+ | EvalCreateResponse.EvalGraderScoreModel
>;
}
export namespace EvalCreateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -377,85 +244,11 @@ export namespace EvalCreateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -465,7 +258,7 @@ export namespace EvalCreateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalRetrieveResponse {
/**
@@ -481,7 +274,10 @@ export interface EvalRetrieveResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalRetrieveResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -507,39 +303,59 @@ export interface EvalRetrieveResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalRetrieveResponse.Python
- | EvalRetrieveResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalRetrieveResponse.EvalGraderTextSimilarity
+ | EvalRetrieveResponse.EvalGraderPython
+ | EvalRetrieveResponse.EvalGraderScoreModel
>;
}
export namespace EvalRetrieveResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -549,85 +365,11 @@ export namespace EvalRetrieveResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -637,7 +379,7 @@ export namespace EvalRetrieveResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalUpdateResponse {
/**
@@ -653,7 +395,10 @@ export interface EvalUpdateResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalUpdateResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -679,39 +424,59 @@ export interface EvalUpdateResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalUpdateResponse.Python
- | EvalUpdateResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalUpdateResponse.EvalGraderTextSimilarity
+ | EvalUpdateResponse.EvalGraderPython
+ | EvalUpdateResponse.EvalGraderScoreModel
>;
}
export namespace EvalUpdateResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -721,85 +486,11 @@ export namespace EvalUpdateResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -809,7 +500,7 @@ export namespace EvalUpdateResponse {
*
* - Improve the quality of my chatbot
* - See how well my chatbot handles customer support
- * - Check if o3-mini is better at my usecase than gpt-4o
+ * - Check if o4-mini is better at my usecase than gpt-4o
*/
export interface EvalListResponse {
/**
@@ -825,7 +516,10 @@ export interface EvalListResponse {
/**
* Configuration of data sources used in runs of the evaluation.
*/
- data_source_config: EvalCustomDataSourceConfig | EvalStoredCompletionsDataSourceConfig;
+ data_source_config:
+ | EvalCustomDataSourceConfig
+ | EvalListResponse.Logs
+ | EvalStoredCompletionsDataSourceConfig;
/**
* Set of 16 key-value pairs that can be attached to an object. This can be useful
@@ -851,39 +545,59 @@ export interface EvalListResponse {
* A list of testing criteria.
*/
testing_criteria: Array<
- | EvalLabelModelGrader
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
- | EvalListResponse.Python
- | EvalListResponse.ScoreModel
+ | GraderModelsAPI.LabelModelGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalListResponse.EvalGraderTextSimilarity
+ | EvalListResponse.EvalGraderPython
+ | EvalListResponse.EvalGraderScoreModel
>;
}
export namespace EvalListResponse {
/**
- * A PythonGrader object that runs a python script on the input.
+ * A LogsDataSourceConfig which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc. The
+ * schema returned by this data source config is used to defined what variables are
+ * available in your evals. `item` and `sample` are both defined when using this
+ * data source config.
*/
- export interface Python {
+ export interface Logs {
/**
- * The name of the grader.
+ * The json schema for the run data source items. Learn how to build JSON schemas
+ * [here](https://json-schema.org/).
*/
- name: string;
+ schema: Record<string, unknown>;
/**
- * The source code of the python script.
+ * The type of data source. Always `logs`.
*/
- source: string;
+ type: 'logs';
/**
- * The object type, which is always `python`.
+ * Set of 16 key-value pairs that can be attached to an object. This can be useful
+ * for storing additional information about the object in a structured format, and
+ * querying for objects via API or the dashboard.
+ *
+ * Keys are strings with a maximum length of 64 characters. Values are strings with
+ * a maximum length of 512 characters.
*/
- type: 'python';
+ metadata?: Shared.Metadata | null;
+ }
+ /**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+ export interface EvalGraderTextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface EvalGraderPython extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -893,85 +607,11 @@ export namespace EvalListResponse {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface EvalGraderScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -985,17 +625,21 @@ export interface EvalDeleteResponse {
export interface EvalCreateParams {
/**
- * The configuration for the data source used for the evaluation runs.
+ * The configuration for the data source used for the evaluation runs. Dictates the
+ * schema of the data used in the evaluation.
*/
- data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs;
+ data_source_config: EvalCreateParams.Custom | EvalCreateParams.Logs | EvalCreateParams.StoredCompletions;
/**
- * A list of graders for all eval runs in this group.
+ * A list of graders for all eval runs in this group. Graders can reference
+ * variables in the data source using double curly braces notation, like
+ * `{{item.variable_name}}`. To reference the model's output, use the `sample`
+ * namespace (ie, `{{sample.output_text}}`).
*/
testing_criteria: Array<
| EvalCreateParams.LabelModel
- | EvalStringCheckGrader
- | EvalTextSimilarityGrader
+ | GraderModelsAPI.StringCheckGrader
+ | EvalCreateParams.TextSimilarity
| EvalCreateParams.Python
| EvalCreateParams.ScoreModel
>;
@@ -1044,9 +688,8 @@ export namespace EvalCreateParams {
}
/**
- * A data source config which specifies the metadata property of your stored
- * completions query. This is usually metadata like `usecase=chatbot` or
- * `prompt-version=v2`, etc.
+ * A data source config which specifies the metadata property of your logs query.
+ * This is usually metadata like `usecase=chatbot` or `prompt-version=v2`, etc.
*/
export interface Logs {
/**
@@ -1060,6 +703,21 @@ export namespace EvalCreateParams {
metadata?: Record<string, unknown>;
}
+ /**
+ * @deprecated Deprecated in favor of LogsDataSourceConfig.
+ */
+ export interface StoredCompletions {
+ /**
+ * The type of data source. Always `stored_completions`.
+ */
+ type: 'stored_completions';
+
+ /**
+ * Metadata filters for the stored completions data source.
+ */
+ metadata?: Record<string, unknown>;
+ }
+
/**
* A LabelModelGrader object which uses a model to assign labels to each item in
* the evaluation.
@@ -1067,7 +725,7 @@ export namespace EvalCreateParams {
export interface LabelModel {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
input: Array<LabelModel.SimpleInputMessage | LabelModel.EvalItem>;
@@ -1154,29 +812,19 @@ export namespace EvalCreateParams {
}
/**
- * A PythonGrader object that runs a python script on the input.
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
*/
- export interface Python {
+ export interface TextSimilarity extends GraderModelsAPI.TextSimilarityGrader {
/**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The source code of the python script.
- */
- source: string;
-
- /**
- * The object type, which is always `python`.
- */
- type: 'python';
-
- /**
- * The image tag to use for the python script.
+ * The threshold for the score.
*/
- image_tag?: string;
+ pass_threshold: number;
+ }
+ /**
+ * A PythonGrader object that runs a python script on the input.
+ */
+ export interface Python extends GraderModelsAPI.PythonGrader {
/**
* The threshold for the score.
*/
@@ -1186,85 +834,11 @@ export namespace EvalCreateParams {
/**
* A ScoreModelGrader object that uses a model to assign a score to the input.
*/
- export interface ScoreModel {
- /**
- * The input text. This may include template strings.
- */
- input: Array<ScoreModel.Input>;
-
- /**
- * The model to use for the evaluation.
- */
- model: string;
-
- /**
- * The name of the grader.
- */
- name: string;
-
- /**
- * The object type, which is always `score_model`.
- */
- type: 'score_model';
-
+ export interface ScoreModel extends GraderModelsAPI.ScoreModelGrader {
/**
* The threshold for the score.
*/
pass_threshold?: number;
-
- /**
- * The range of the score. Defaults to `[0, 1]`.
- */
- range?: Array<number>;
-
- /**
- * The sampling parameters for the model.
- */
- sampling_params?: unknown;
- }
-
- export namespace ScoreModel {
- /**
- * A message input to the model with a role indicating instruction following
- * hierarchy. Instructions given with the `developer` or `system` role take
- * precedence over instructions given with the `user` role. Messages with the
- * `assistant` role are presumed to have been generated by the model in previous
- * interactions.
- */
- export interface Input {
- /**
- * Text inputs to the model - can contain template strings.
- */
- content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
-
- /**
- * The role of the message input. One of `user`, `assistant`, `system`, or
- * `developer`.
- */
- role: 'user' | 'assistant' | 'system' | 'developer';
-
- /**
- * The type of the message input. Always `message`.
- */
- type?: 'message';
- }
-
- export namespace Input {
- /**
- * A text output from the model.
- */
- export interface OutputText {
- /**
- * The text output from the model.
- */
- text: string;
-
- /**
- * The type of the output text. Always `output_text`.
- */
- type: 'output_text';
- }
- }
}
}
@@ -1306,10 +880,7 @@ Evals.RunListResponsesPage = RunListResponsesPage;
export declare namespace Evals {
export {
type EvalCustomDataSourceConfig as EvalCustomDataSourceConfig,
- type EvalLabelModelGrader as EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig as EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader as EvalStringCheckGrader,
- type EvalTextSimilarityGrader as EvalTextSimilarityGrader,
type EvalCreateResponse as EvalCreateResponse,
type EvalRetrieveResponse as EvalRetrieveResponse,
type EvalUpdateResponse as EvalUpdateResponse,
diff --git src/resources/evals/index.ts src/resources/evals/index.ts
index a246fe4e7..b2627fbf3 100644
--- src/resources/evals/index.ts
+++ src/resources/evals/index.ts
@@ -4,10 +4,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
diff --git src/resources/evals/runs/runs.ts src/resources/evals/runs/runs.ts
index 50c07a514..e761e2160 100644
--- src/resources/evals/runs/runs.ts
+++ src/resources/evals/runs/runs.ts
@@ -19,7 +19,9 @@ export class Runs extends APIResource {
outputItems: OutputItemsAPI.OutputItems = new OutputItemsAPI.OutputItems(this._client);
/**
- * Create a new evaluation run. This is the endpoint that will kick off grading.
+ * Kicks off a new run for a given evaluation, specifying the data source, and what
+ * model configuration to use to test. The datasource will be validated against the
+ * schema specified in the config of the evaluation.
*/
create(
evalId: string,
@@ -85,7 +87,7 @@ export class RunListResponsesPage extends CursorPage<RunListResponse> {}
*/
export interface CreateEvalCompletionsRunDataSource {
/**
- * A StoredCompletionsRunDataSource configuration describing a set of filters
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalCompletionsRunDataSource.FileContent
@@ -97,6 +99,12 @@ export interface CreateEvalCompletionsRunDataSource {
*/
type: 'completions';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalCompletionsRunDataSource.Template
| CreateEvalCompletionsRunDataSource.ItemReference;
@@ -185,7 +193,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<ResponsesAPI.EasyInputMessage | Template.Message>;
@@ -241,7 +249,7 @@ export namespace CreateEvalCompletionsRunDataSource {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"
*/
item_reference: string;
@@ -279,6 +287,9 @@ export namespace CreateEvalCompletionsRunDataSource {
* eval
*/
export interface CreateEvalJSONLRunDataSource {
+ /**
+ * Determines what populates the `item` namespace in the data source.
+ */
source: CreateEvalJSONLRunDataSource.FileContent | CreateEvalJSONLRunDataSource.FileID;
/**
@@ -356,7 +367,7 @@ export interface RunCreateResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCreateResponse.Completions;
+ | RunCreateResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -423,28 +434,34 @@ export namespace RunCreateResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -486,12 +503,6 @@ export namespace RunCreateResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -505,14 +516,8 @@ export namespace RunCreateResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -539,6 +544,11 @@ export namespace RunCreateResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -553,7 +563,7 @@ export namespace RunCreateResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -621,7 +631,7 @@ export namespace RunCreateResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -749,7 +759,7 @@ export interface RunRetrieveResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunRetrieveResponse.Completions;
+ | RunRetrieveResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -816,28 +826,34 @@ export namespace RunRetrieveResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -879,12 +895,6 @@ export namespace RunRetrieveResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -898,14 +908,8 @@ export namespace RunRetrieveResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -932,6 +936,11 @@ export namespace RunRetrieveResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -946,7 +955,7 @@ export namespace RunRetrieveResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1014,7 +1023,7 @@ export namespace RunRetrieveResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1139,10 +1148,7 @@ export interface RunListResponse {
/**
* Information about the run's data source.
*/
- data_source:
- | CreateEvalJSONLRunDataSource
- | CreateEvalCompletionsRunDataSource
- | RunListResponse.Completions;
+ data_source: CreateEvalJSONLRunDataSource | CreateEvalCompletionsRunDataSource | RunListResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1209,28 +1215,34 @@ export namespace RunListResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1272,12 +1284,6 @@ export namespace RunListResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1291,14 +1297,8 @@ export namespace RunListResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1325,6 +1325,11 @@ export namespace RunListResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1339,7 +1344,7 @@ export namespace RunListResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1407,7 +1412,7 @@ export namespace RunListResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1543,7 +1548,7 @@ export interface RunCancelResponse {
data_source:
| CreateEvalJSONLRunDataSource
| CreateEvalCompletionsRunDataSource
- | RunCancelResponse.Completions;
+ | RunCancelResponse.Responses;
/**
* An object representing an error response from the Eval API.
@@ -1610,28 +1615,34 @@ export namespace RunCancelResponse {
/**
* A ResponsesRunDataSource object describing a model sampling configuration.
*/
- export interface Completions {
+ export interface Responses {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
- source: Completions.FileContent | Completions.FileID | Completions.Responses;
+ source: Responses.FileContent | Responses.FileID | Responses.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
- input_messages?: Completions.Template | Completions.ItemReference;
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
+ input_messages?: Responses.Template | Responses.ItemReference;
/**
* The name of the model to use for generating completions (e.g. "o3-mini").
*/
model?: string;
- sampling_params?: Completions.SamplingParams;
+ sampling_params?: Responses.SamplingParams;
}
- export namespace Completions {
+ export namespace Responses {
export interface FileContent {
/**
* The content of the jsonl file.
@@ -1673,12 +1684,6 @@ export namespace RunCancelResponse {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -1692,14 +1697,8 @@ export namespace RunCancelResponse {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -1726,6 +1725,11 @@ export namespace RunCancelResponse {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -1740,7 +1744,7 @@ export namespace RunCancelResponse {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -1808,7 +1812,7 @@ export namespace RunCancelResponse {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
@@ -1947,7 +1951,7 @@ export namespace RunCreateParams {
*/
export interface CreateEvalResponsesRunDataSource {
/**
- * A EvalResponsesSource object describing a run data source configuration.
+ * Determines what populates the `item` namespace in this run's data source.
*/
source:
| CreateEvalResponsesRunDataSource.FileContent
@@ -1955,10 +1959,16 @@ export namespace RunCreateParams {
| CreateEvalResponsesRunDataSource.Responses;
/**
- * The type of run data source. Always `completions`.
+ * The type of run data source. Always `responses`.
*/
- type: 'completions';
+ type: 'responses';
+ /**
+ * Used when sampling from a model. Dictates the structure of the messages passed
+ * into the model. Can either be a reference to a prebuilt trajectory (ie,
+ * `item.input_trajectory`), or a template with variable references to the `item`
+ * namespace.
+ */
input_messages?:
| CreateEvalResponsesRunDataSource.Template
| CreateEvalResponsesRunDataSource.ItemReference;
@@ -2013,12 +2023,6 @@ export namespace RunCreateParams {
*/
type: 'responses';
- /**
- * Whether to allow parallel tool calls. This is a query parameter used to select
- * responses.
- */
- allow_parallel_tool_calls?: boolean | null;
-
/**
* Only include items created after this timestamp (inclusive). This is a query
* parameter used to select responses.
@@ -2032,14 +2036,8 @@ export namespace RunCreateParams {
created_before?: number | null;
/**
- * Whether the response has tool calls. This is a query parameter used to select
- * responses.
- */
- has_tool_calls?: boolean | null;
-
- /**
- * Optional search string for instructions. This is a query parameter used to
- * select responses.
+ * Optional string to search the 'instructions' field. This is a query parameter
+ * used to select responses.
*/
instructions_search?: string | null;
@@ -2066,6 +2064,11 @@ export namespace RunCreateParams {
*/
temperature?: number | null;
+ /**
+ * List of tool names. This is a query parameter used to select responses.
+ */
+ tools?: Array<string> | null;
+
/**
* Nucleus sampling parameter. This is a query parameter used to select responses.
*/
@@ -2080,7 +2083,7 @@ export namespace RunCreateParams {
export interface Template {
/**
* A list of chat messages forming the prompt or context. May include variable
- * references to the "item" namespace, ie {{item.name}}.
+ * references to the `item` namespace, ie {{item.name}}.
*/
template: Array<Template.ChatMessage | Template.EvalItem>;
@@ -2148,7 +2151,7 @@ export namespace RunCreateParams {
export interface ItemReference {
/**
- * A reference to a variable in the "item" namespace. Ie, "item.name"
+ * A reference to a variable in the `item` namespace. Ie, "item.name"
*/
item_reference: string;
diff --git a/src/resources/fine-tuning/alpha.ts b/src/resources/fine-tuning/alpha.ts
new file mode 100644
index 000000000..446b6431e
--- /dev/null
+++ src/resources/fine-tuning/alpha.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './alpha/index';
diff --git a/src/resources/fine-tuning/alpha/alpha.ts b/src/resources/fine-tuning/alpha/alpha.ts
new file mode 100644
index 000000000..77d695195
--- /dev/null
+++ src/resources/fine-tuning/alpha/alpha.ts
@@ -0,0 +1,27 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as GradersAPI from './graders';
+import {
+ GraderRunParams,
+ GraderRunResponse,
+ GraderValidateParams,
+ GraderValidateResponse,
+ Graders,
+} from './graders';
+
+export class Alpha extends APIResource {
+ graders: GradersAPI.Graders = new GradersAPI.Graders(this._client);
+}
+
+Alpha.Graders = Graders;
+
+export declare namespace Alpha {
+ export {
+ Graders as Graders,
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/graders.ts b/src/resources/fine-tuning/alpha/graders.ts
new file mode 100644
index 000000000..a9ef57f71
--- /dev/null
+++ src/resources/fine-tuning/alpha/graders.ts
@@ -0,0 +1,168 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../../resource';
+import * as Core from '../../../core';
+import * as GraderModelsAPI from '../../graders/grader-models';
+
+export class Graders extends APIResource {
+ /**
+ * Run a grader.
+ *
+ * @example
+ * ```ts
+ * const response = await client.fineTuning.alpha.graders.run({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * model_sample: 'model_sample',
+ * reference_answer: 'string',
+ * });
+ * ```
+ */
+ run(body: GraderRunParams, options?: Core.RequestOptions): Core.APIPromise<GraderRunResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/run', { body, ...options });
+ }
+
+ /**
+ * Validate a grader.
+ *
+ * @example
+ * ```ts
+ * const response =
+ * await client.fineTuning.alpha.graders.validate({
+ * grader: {
+ * input: 'input',
+ * name: 'name',
+ * operation: 'eq',
+ * reference: 'reference',
+ * type: 'string_check',
+ * },
+ * });
+ * ```
+ */
+ validate(
+ body: GraderValidateParams,
+ options?: Core.RequestOptions,
+ ): Core.APIPromise<GraderValidateResponse> {
+ return this._client.post('/fine_tuning/alpha/graders/validate', { body, ...options });
+ }
+}
+
+export interface GraderRunResponse {
+ metadata: GraderRunResponse.Metadata;
+
+ model_grader_token_usage_per_model: Record<string, unknown>;
+
+ reward: number;
+
+ sub_rewards: Record<string, unknown>;
+}
+
+export namespace GraderRunResponse {
+ export interface Metadata {
+ errors: Metadata.Errors;
+
+ execution_time: number;
+
+ name: string;
+
+ sampled_model_name: string | null;
+
+ scores: Record<string, unknown>;
+
+ token_usage: number | null;
+
+ type: string;
+ }
+
+ export namespace Metadata {
+ export interface Errors {
+ formula_parse_error: boolean;
+
+ invalid_variable_error: boolean;
+
+ model_grader_parse_error: boolean;
+
+ model_grader_refusal_error: boolean;
+
+ model_grader_server_error: boolean;
+
+ model_grader_server_error_details: string | null;
+
+ other_error: boolean;
+
+ python_grader_runtime_error: boolean;
+
+ python_grader_runtime_error_details: string | null;
+
+ python_grader_server_error: boolean;
+
+ python_grader_server_error_type: string | null;
+
+ sample_parse_error: boolean;
+
+ truncated_observation_error: boolean;
+
+ unresponsive_reward_error: boolean;
+ }
+ }
+}
+
+export interface GraderValidateResponse {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader?:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export interface GraderRunParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The model sample to be evaluated.
+ */
+ model_sample: string;
+
+ /**
+ * The reference answer for the evaluation.
+ */
+ reference_answer: string | unknown | Array<unknown> | number;
+}
+
+export interface GraderValidateParams {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+}
+
+export declare namespace Graders {
+ export {
+ type GraderRunResponse as GraderRunResponse,
+ type GraderValidateResponse as GraderValidateResponse,
+ type GraderRunParams as GraderRunParams,
+ type GraderValidateParams as GraderValidateParams,
+ };
+}
diff --git a/src/resources/fine-tuning/alpha/index.ts b/src/resources/fine-tuning/alpha/index.ts
new file mode 100644
index 000000000..47b229bc3
--- /dev/null
+++ src/resources/fine-tuning/alpha/index.ts
@@ -0,0 +1,10 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export { Alpha } from './alpha';
+export {
+ Graders,
+ type GraderRunResponse,
+ type GraderValidateResponse,
+ type GraderRunParams,
+ type GraderValidateParams,
+} from './graders';
diff --git src/resources/fine-tuning/checkpoints/permissions.ts src/resources/fine-tuning/checkpoints/permissions.ts
index e808b2001..dc25bab7f 100644
--- src/resources/fine-tuning/checkpoints/permissions.ts
+++ src/resources/fine-tuning/checkpoints/permissions.ts
@@ -11,6 +11,17 @@ export class Permissions extends APIResource {
*
* This enables organization owners to share fine-tuned models with other projects
* in their organization.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const permissionCreateResponse of client.fineTuning.checkpoints.permissions.create(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * { project_ids: ['string'] },
+ * )) {
+ * // ...
+ * }
+ * ```
*/
create(
fineTunedModelCheckpoint: string,
@@ -29,6 +40,14 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to view all permissions for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(
fineTunedModelCheckpoint: string,
@@ -58,6 +77,15 @@ export class Permissions extends APIResource {
*
* Organization owners can use this endpoint to delete a permission for a
* fine-tuned model checkpoint.
+ *
+ * @example
+ * ```ts
+ * const permission =
+ * await client.fineTuning.checkpoints.permissions.del(
+ * 'ft:gpt-4o-mini-2024-07-18:org:weather:B7R9VjQd',
+ * 'cp_zc4Q7MP6XxulcVzj4MZdwsAB',
+ * );
+ * ```
*/
del(
fineTunedModelCheckpoint: string,
diff --git src/resources/fine-tuning/fine-tuning.ts src/resources/fine-tuning/fine-tuning.ts
index 9b0a01992..8fb54983b 100644
--- src/resources/fine-tuning/fine-tuning.ts
+++ src/resources/fine-tuning/fine-tuning.ts
@@ -1,6 +1,18 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
import { APIResource } from '../../resource';
+import * as MethodsAPI from './methods';
+import {
+ DpoHyperparamet,ers,
+ DpoMethod,
+ Methods,
+ ReinforcementHyperparameters,
+ ReinforcementMethod,
+ SupervisedHyperparameters,
+ SupervisedMethod,
+} from './methods';
+import * as AlphaAPI from './alpha/alpha';
+import { Alpha } from './alpha/alpha';
import * as CheckpointsAPI from './checkpoints/checkpoints';
import { Checkpoints } from './checkpoints/checkpoints';
import * as JobsAPI from './jobs/jobs';
@@ -19,16 +31,30 @@ import {
} from './jobs/jobs';
export class FineTuning extends APIResource {
+ methods: MethodsAPI.Methods = new MethodsAPI.Methods(this._client);
jobs: JobsAPI.Jobs = new JobsAPI.Jobs(this._client);
checkpoints: CheckpointsAPI.Checkpoints = new CheckpointsAPI.Checkpoints(this._client);
+ alpha: AlphaAPI.Alpha = new AlphaAPI.Alpha(this._client);
}
+FineTuning.Methods = Methods;
FineTuning.Jobs = Jobs;
FineTuning.FineTuningJobsPage = FineTuningJobsPage;
FineTuning.FineTuningJobEventsPage = FineTuningJobEventsPage;
FineTuning.Checkpoints = Checkpoints;
+FineTuning.Alpha = Alpha;
export declare namespace FineTuning {
+ export {
+ Methods as Methods,
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+
export {
Jobs as Jobs,
type FineTuningJob as FineTuningJob,
@@ -44,4 +70,6 @@ export declare namespace FineTuning {
};
export { Checkpoints as Checkpoints };
+
+ export { Alpha as Alpha };
}
diff --git src/resources/fine-tuning/index.ts src/resources/fine-tuning/index.ts
index d23161c62..878ac402d 100644
--- src/resources/fine-tuning/index.ts
+++ src/resources/fine-tuning/index.ts
@@ -1,5 +1,6 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+export { Alpha } from './alpha/index';
export { Checkpoints } from './checkpoints/index';
export { FineTuning } from './fine-tuning';
export {
@@ -15,3 +16,12 @@ export {
type JobListParams,
type JobListEventsParams,
} from './jobs/index';
+export {
+ Methods,
+ type DpoHyperparameters,
+ type DpoMethod,
+ type ReinforcementHyperparameters,
+ type ReinforcementMethod,
+ type SupervisedHyperparameters,
+ type SupervisedMethod,
+} from './methods';
diff --git src/resources/fine-tuning/jobs/checkpoints.ts src/resources/fine-tuning/jobs/checkpoints.ts
index b3018ac5f..10902e715 100644
--- src/resources/fine-tuning/jobs/checkpoints.ts
+++ src/resources/fine-tuning/jobs/checkpoints.ts
@@ -8,6 +8,16 @@ import { CursorPage, type CursorPageParams } from '../../../pagination';
export class Checkpoints extends APIResource {
/**
* List checkpoints for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobCheckpoint of client.fineTuning.jobs.checkpoints.list(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
fineTuningJobId: string,
diff --git src/resources/fine-tuning/jobs/jobs.ts src/resources/fine-tuning/jobs/jobs.ts
index 9be03c302..cc5f55e9a 100644
--- src/resources/fine-tuning/jobs/jobs.ts
+++ src/resources/fine-tuning/jobs/jobs.ts
@@ -3,6 +3,7 @@
import { APIResource } from '../../../resource';
import { isRequestOptions } from '../../../core';
import * as Core from '../../../core';
+import * as MethodsAPI from '../methods';
import * as CheckpointsAPI from './checkpoints';
import {
CheckpointListParams,
@@ -23,6 +24,14 @@ export class Jobs extends APIResource {
* of the fine-tuned models once complete.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.create({
+ * model: 'gpt-4o-mini',
+ * training_file: 'file-abc123',
+ * });
+ * ```
*/
create(body: JobCreateParams, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post('/fine_tuning/jobs', { body, ...options });
@@ -32,6 +41,13 @@ export class Jobs extends APIResource {
* Get info about a fine-tuning job.
*
* [Learn more about fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.retrieve(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
retrieve(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.get(`/fine_tuning/jobs/${fineTuningJobId}`, options);
@@ -39,6 +55,14 @@ export class Jobs extends APIResource {
/**
* List your organization's fine-tuning jobs
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJob of client.fineTuning.jobs.list()) {
+ * // ...
+ * }
+ * ```
*/
list(
query?: JobListParams,
@@ -57,6 +81,13 @@ export class Jobs extends APIResource {
/**
* Immediately cancel a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.cancel(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
*/
cancel(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/cancel`, options);
@@ -64,6 +95,16 @@ export class Jobs extends APIResource {
/**
* Get status updates for a fine-tuning job.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const fineTuningJobEvent of client.fineTuning.jobs.listEvents(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
listEvents(
fineTuningJobId: string,
@@ -87,6 +128,34 @@ export class Jobs extends APIResource {
...options,
});
}
+
+ /**
+ * Pause a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.pause(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ pause(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/pause`, options);
+ }
+
+ /**
+ * Resume a fine-tune job.
+ *
+ * @example
+ * ```ts
+ * const fineTuningJob = await client.fineTuning.jobs.resume(
+ * 'ft-AF1WoRqd3aJAHsqc9NY7iL8F',
+ * );
+ * ```
+ */
+ resume(fineTuningJobId: string, options?: Core.RequestOptions): Core.APIPromise<FineTuningJob> {
+ return this._client.post(`/fine_tuning/jobs/${fineTuningJobId}/resume`, options);
+ }
}
export class FineTuningJobsPage extends CursorPage<FineTuningJob> {}
@@ -233,7 +302,7 @@ export namespace FineTuningJob {
* Number of examples in each batch. A larger batch size means that model
* parameters are updated less frequently, but with lower variance.
*/
- batch_size?: 'auto' | number;
+ batch_size?: unknown | 'auto' | number | null;
/**
* Scaling factor for the learning rate. A smaller learning rate may be useful to
@@ -253,97 +322,24 @@ export namespace FineTuningJob {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
+ * Configuration for the DPO fine-tuning method.
*/
- supervised?: Method.Supervised;
+ dpo?: MethodsAPI.DpoMethod;
/**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the reinforcement fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
-
- export namespace Method {
- /**
- * Configuration for the DPO fine-tuning method.
- */
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
@@ -466,8 +462,9 @@ export interface JobCreateParams {
training_file: string;
/**
- * The hyperparameters used for the fine-tuning job. This value is now deprecated
- * in favor of `method`, and should be passed in under the `method` parameter.
+ * @deprecated The hyperparameters used for the fine-tuning job. This value is now
+ * deprecated in favor of `method`, and should be passed in under the `method`
+ * parameter.
*/
hyperparameters?: JobCreateParams.Hyperparameters;
@@ -596,97 +593,24 @@ export namespace JobCreateParams {
*/
export interface Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * The type of method. Is either `supervised`, `dpo`, or `reinforcement`.
*/
- dpo?: Method.Dpo;
+ type: 'supervised' | 'dpo' | 'reinforcement';
/**
- * Configuration for the supervised fine-tuning method.
- */
- supervised?: Method.Supervised;
-
- /**
- * The type of method. Is either `supervised` or `dpo`.
+ * Configuration for the DPO fine-tuning method.
*/
- type?: 'supervised' | 'dpo';
- }
+ dpo?: MethodsAPI.DpoMethod;
- export namespace Method {
/**
- * Configuration for the DPO fine-tuning method.
+ * Configuration for the reinforcement fine-tuning method.
*/
- export interface Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Dpo.Hyperparameters;
- }
-
- export namespace Dpo {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * The beta value for the DPO method. A higher beta value will increase the weight
- * of the penalty between the policy and reference model.
- */
- beta?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ reinforcement?: MethodsAPI.ReinforcementMethod;
/**
* Configuration for the supervised fine-tuning method.
*/
- export interface Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- hyperparameters?: Supervised.Hyperparameters;
- }
-
- export namespace Supervised {
- /**
- * The hyperparameters used for the fine-tuning job.
- */
- export interface Hyperparameters {
- /**
- * Number of examples in each batch. A larger batch size means that model
- * parameters are updated less frequently, but with lower variance.
- */
- batch_size?: 'auto' | number;
-
- /**
- * Scaling factor for the learning rate. A smaller learning rate may be useful to
- * avoid overfitting.
- */
- learning_rate_multiplier?: 'auto' | number;
-
- /**
- * The number of epochs to train the model for. An epoch refers to one full cycle
- * through the training dataset.
- */
- n_epochs?: 'auto' | number;
- }
- }
+ supervised?: MethodsAPI.SupervisedMethod;
}
}
diff --git a/src/resources/fine-tuning/methods.ts b/src/resources/fine-tuning/methods.ts
new file mode 100644
index 000000000..aa459c74c
--- /dev/null
+++ src/resources/fine-tuning/methods.ts
@@ -0,0 +1,152 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from '../graders/grader-models';
+
+export class Methods extends APIResource {}
+
+/**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+export interface DpoHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * The beta value for the DPO method. A higher beta value will increase the weight
+ * of the penalty between the policy and reference model.
+ */
+ beta?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the DPO fine-tuning method.
+ */
+export interface DpoMethod {
+ /**
+ * The hyperparameters used for the DPO fine-tuning job.
+ */
+ hyperparameters?: DpoHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+export interface ReinforcementHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Multiplier on amount of compute used for exploring search space during training.
+ */
+ compute_multiplier?: 'auto' | number;
+
+ /**
+ * The number of training steps between evaluation runs.
+ */
+ eval_interval?: 'auto' | number;
+
+ /**
+ * Number of evaluation samples to generate per training step.
+ */
+ eval_samples?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+
+ /**
+ * Level of reasoning effort.
+ */
+ reasoning_effort?: 'default' | 'low' | 'medium' | 'high';
+}
+
+/**
+ * Configuration for the reinforcement fine-tuning method.
+ */
+export interface ReinforcementMethod {
+ /**
+ * The grader used for the fine-tuning job.
+ */
+ grader:
+ | GraderModelsAPI.StringCheckGrader
+ | GraderModelsAPI.TextSimilarityGrader
+ | GraderModelsAPI.PythonGrader
+ | GraderModelsAPI.ScoreModelGrader
+ | GraderModelsAPI.MultiGrader;
+
+ /**
+ * The hyperparameters used for the reinforcement fine-tuning job.
+ */
+ hyperparameters?: ReinforcementHyperparameters;
+}
+
+/**
+ * The hyperparameters used for the fine-tuning job.
+ */
+export interface SupervisedHyperparameters {
+ /**
+ * Number of examples in each batch. A larger batch size means that model
+ * parameters are updated less frequently, but with lower variance.
+ */
+ batch_size?: 'auto' | number;
+
+ /**
+ * Scaling factor for the learning rate. A smaller learning rate may be useful to
+ * avoid overfitting.
+ */
+ learning_rate_multiplier?: 'auto' | number;
+
+ /**
+ * The number of epochs to train the model for. An epoch refers to one full cycle
+ * through the training dataset.
+ */
+ n_epochs?: 'auto' | number;
+}
+
+/**
+ * Configuration for the supervised fine-tuning method.
+ */
+export interface SupervisedMethod {
+ /**
+ * The hyperparameters used for the fine-tuning job.
+ */
+ hyperparameters?: SupervisedHyperparameters;
+}
+
+export declare namespace Methods {
+ export {
+ type DpoHyperparameters as DpoHyperparameters,
+ type DpoMethod as DpoMethod,
+ type ReinforcementHyperparameters as ReinforcementHyperparameters,
+ type ReinforcementMethod as ReinforcementMethod,
+ type SupervisedHyperparameters as SupervisedHyperparameters,
+ type SupervisedMethod as SupervisedMethod,
+ };
+}
diff --git a/src/resources/graders.ts b/src/resources/graders.ts
new file mode 100644
index 000000000..2ea9aa959
--- /dev/null
+++ src/resources/graders.ts
@@ -0,0 +1,3 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export * from './graders/index';
diff --git a/src/resources/graders/grader-models.ts b/src/resources/graders/grader-models.ts
new file mode 100644
index 000000000..d2c335300
--- /dev/null
+++ src/resources/graders/grader-models.ts
@@ -0,0 +1,296 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as ResponsesAPI from '../responses/responses';
+
+export class GraderModels extends APIResource {}
+
+/**
+ * A LabelModelGrader object which uses a model to assign labels to each item in
+ * the evaluation.
+ */
+export interface LabelModelGrader {
+ input: Array<LabelModelGrader.Input>;
+
+ /**
+ * The labels to assign to each item in the evaluation.
+ */
+ labels: Array<string>;
+
+ /**
+ * The model to use for the evaluation. Must support structured outputs.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The labels that indicate a passing result. Must be a subset of labels.
+ */
+ passing_labels: Array<string>;
+
+ /**
+ * The object type, which is always `label_model`.
+ */
+ type: 'label_model';
+}
+
+export namespace LabelModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A MultiGrader object combines the output of multiple graders to produce a single
+ * score.
+ */
+export interface MultiGrader {
+ /**
+ * A formula to calculate the output based on grader results.
+ */
+ calculate_output: string;
+
+ graders: Record<
+ string,
+ StringCheckGrader | TextSimilarityGrader | PythonGrader | ScoreModelGrader | LabelModelGrader
+ >;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `multi`.
+ */
+ type: 'multi';
+}
+
+/**
+ * A PythonGrader object that runs a python script on the input.
+ */
+export interface PythonGrader {
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The source code of the python script.
+ */
+ source: string;
+
+ /**
+ * The object type, which is always `python`.
+ */
+ type: 'python';
+
+ /**
+ * The image tag to use for the python script.
+ */
+ image_tag?: string;
+}
+
+/**
+ * A ScoreModelGrader object that uses a model to assign a score to the input.
+ */
+export interface ScoreModelGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: Array<ScoreModelGrader.Input>;
+
+ /**
+ * The model to use for the evaluation.
+ */
+ model: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The object type, which is always `score_model`.
+ */
+ type: 'score_model';
+
+ /**
+ * The range of the score. Defaults to `[0, 1]`.
+ */
+ range?: Array<number>;
+
+ /**
+ * The sampling parameters for the model.
+ */
+ sampling_params?: unknown;
+}
+
+export namespace ScoreModelGrader {
+ /**
+ * A message input to the model with a role indicating instruction following
+ * hierarchy. Instructions given with the `developer` or `system` role take
+ * precedence over instructions given with the `user` role. Messages with the
+ * `assistant` role are presumed to have been generated by the model in previous
+ * interactions.
+ */
+ export interface Input {
+ /**
+ * Text inputs to the model - can contain template strings.
+ */
+ content: string | ResponsesAPI.ResponseInputText | Input.OutputText;
+
+ /**
+ * The role of the message input. One of `user`, `assistant`, `system`, or
+ * `developer`.
+ */
+ role: 'user' | 'assistant' | 'system' | 'developer';
+
+ /**
+ * The type of the message input. Always `message`.
+ */
+ type?: 'message';
+ }
+
+ export namespace Input {
+ /**
+ * A text output from the model.
+ */
+ export interface OutputText {
+ /**
+ * The text output from the model.
+ */
+ text: string;
+
+ /**
+ * The type of the output text. Always `output_text`.
+ */
+ type: 'output_text';
+ }
+ }
+}
+
+/**
+ * A StringCheckGrader object that performs a string comparison between input and
+ * reference using a specified operation.
+ */
+export interface StringCheckGrader {
+ /**
+ * The input text. This may include template strings.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`.
+ */
+ operation: 'eq' | 'ne' | 'like' | 'ilike';
+
+ /**
+ * The reference, text. This may include template strings.
+ */
+ reference: string;
+
+ /**
+ * The object type, which is always `string_check`.
+ */
+ type: 'string_check';
+}
+
+/**
+ * A TextSimilarityGrader object which grades text based on similarity metrics.
+ */
+export interface TextSimilarityGrader {
+ /**
+ * The evaluation metric to use. One of `fuzzy_match`, `bleu`, `gleu`, `meteor`,
+ * `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`.
+ */
+ evaluation_metric:
+ | 'fuzzy_match'
+ | 'bleu'
+ | 'gleu'
+ | 'meteor'
+ | 'rouge_1'
+ | 'rouge_2'
+ | 'rouge_3'
+ | 'rouge_4'
+ | 'rouge_5'
+ | 'rouge_l';
+
+ /**
+ * The text being graded.
+ */
+ input: string;
+
+ /**
+ * The name of the grader.
+ */
+ name: string;
+
+ /**
+ * The text being graded against.
+ */
+ reference: string;
+
+ /**
+ * The type of grader.
+ */
+ type: 'text_similarity';
+}
+
+export declare namespace GraderModels {
+ export {
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/graders.ts b/src/resources/graders/graders.ts
new file mode 100644
index 000000000..de3297450
--- /dev/null
+++ src/resources/graders/graders.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import { APIResource } from '../../resource';
+import * as GraderModelsAPI from './grader-models';
+import {
+ GraderModels,
+ LabelModelGrader,
+ MultiGrader,
+ PythonGrader,
+ ScoreModelGrader,
+ StringCheckGrader,
+ TextSimilarityGrader,
+} from './grader-models';
+
+export class Graders extends APIResource {
+ graderModels: GraderModelsAPI.GraderModels = new GraderModelsAPI.GraderModels(this._client);
+}
+
+Graders.GraderModels = GraderModels;
+
+export declare namespace Graders {
+ export {
+ GraderModels as GraderModels,
+ type LabelModelGrader as LabelModelGrader,
+ type MultiGrader as MultiGrader,
+ type PythonGrader as PythonGrader,
+ type ScoreModelGrader as ScoreModelGrader,
+ type StringCheckGrader as StringCheckGrader,
+ type TextSimilarityGrader as TextSimilarityGrader,
+ };
+}
diff --git a/src/resources/graders/index.ts b/src/resources/graders/index.ts
new file mode 100644
index 000000000..82d557a6a
--- /dev/null
+++ src/resources/graders/index.ts
@@ -0,0 +1,12 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+export {
+ GraderModels,
+ type LabelModelGrader,
+ type MultiGrader,
+ type PythonGrader,
+ type ScoreModelGrader,
+ type StringCheckGrader,
+ type TextSimilarityGrader,
+} from './grader-models';
+export { Graders } from './graders';
diff --git src/resources/images.ts src/resources/images.ts
index de1882d30..c6b14833a 100644
--- src/resources/images.ts
+++ src/resources/images.ts
@@ -6,6 +6,13 @@ import * as Core from '../core';
export class Images extends APIResource {
/**
* Creates a variation of a given image. This endpoint only supports `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.createVariation({
+ * image: fs.createReadStream('otter.png'),
+ * });
+ * ```
*/
createVariation(
body: ImageCreateVariationParams,
@@ -17,6 +24,14 @@ export class Images extends APIResource {
/**
* Creates an edited or extended image given one or more source images and a
* prompt. This endpoint only supports `gpt-image-1` and `dall-e-2`.
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.edit({
+ * image: fs.createReadStream('path/to/file'),
+ * prompt: 'A cute baby sea otter wearing a beret',
+ * });
+ * ```
*/
edit(body: ImageEditParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/edits', Core.multipartFormRequestOptions({ body, ...options }));
@@ -25,6 +40,13 @@ export class Images extends APIResource {
/**
* Creates an image given a prompt.
* [Learn more](https://platform.openai.com/docs/guides/images).
+ *
+ * @example
+ * ```ts
+ * const imagesResponse = await client.images.generate({
+ * prompt: 'A cute baby sea otter',
+ * });
+ * ```
*/
generate(body: ImageGenerateParams, options?: Core.RequestOptions): Core.APIPromise<ImagesResponse> {
return this._client.post('/images/generations', { body, ...options });
@@ -162,10 +184,13 @@ export interface ImageCreateVariationParams {
export interface ImageEditParams {
/**
- * The image(s) to edit. Must be a supported image file or an array of images. For
- * `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
- * 25MB. For `dall-e-2`, you can only provide one image, and it should be a square
- * `png` file less than 4MB.
+ * The image(s) to edit. Must be a supported image file or an array of images.
+ *
+ * For `gpt-image-1`, each image should be a `png`, `webp`, or `jpg` file less than
+ * 25MB. You can provide up to 16 images.
+ *
+ * For `dall-e-2`, you can only provide one image, and it should be a square `png`
+ * file less than 4MB.
*/
image: Core.Uploadable | Array<Core.Uploadable>;
@@ -175,6 +200,17 @@ export interface ImageEditParams {
*/
prompt: string;
+ /**
+ * Allows to set transparency for the background of the generated image(s). This
+ * parameter is only supported for `gpt-image-1`. Must be one of `transparent`,
+ * `opaque` or `auto` (default value). When `auto` is used, the model will
+ * automatically determine the best background for the image.
+ *
+ * If `transparent`, the output format needs to support transparency, so it should
+ * be set to either `png` (default value) or `webp`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto' | null;
+
/**
* An additional image whose fully transparent areas (e.g. where alpha is zero)
* indicate where `image` should be edited. If there are multiple images provided,
@@ -215,7 +251,7 @@ export interface ImageEditParams {
* (landscape), `1024x1536` (portrait), or `auto` (default value) for
* `gpt-image-1`, and one of `256x256`, `512x512`, or `1024x1024` for `dall-e-2`.
*/
- size?: '256x256' | '512x512' | '1024x1024' | null;
+ size?: '256x256' | '512x512' | '1024x1024' | '1536x1024' | '1024x1536' | 'auto' | null;
/**
* A unique identifier representing your end-user, which can help OpenAI to monitor
diff --git src/resources/index.ts src/resources/index.ts
index 0d8ec9220..d1f816a84 100644
--- src/resources/index.ts
+++ src/resources/index.ts
@@ -22,6 +22,15 @@ export {
type CompletionCreateParamsNonStreaming,
type CompletionCreateParamsStreaming,
} from './completions';
+export {
+ ContainerListResponsesPage,
+ Containers,
+ type ContainerCreateResponse,
+ type ContainerRetrieveResponse,
+ type ContainerListResponse,
+ type ContainerCreateParams,
+ type ContainerListParams,
+} from './containers/containers';
export {
Embeddings,
type CreateEmbeddingResponse,
@@ -33,10 +42,7 @@ export {
EvalListResponsesPage,
Evals,
type EvalCustomDataSourceConfig,
- type EvalLabelModelGrader,
type EvalStoredCompletionsDataSourceConfig,
- type EvalStringCheckGrader,
- type EvalTextSimilarityGrader,
type EvalCreateResponse,
type EvalRetrieveResponse,
type EvalUpdateResponse,
@@ -57,6 +63,7 @@ export {
type FileListParams,
} from './files';
export { FineTuning } from './fine-tuning/fine-tuning';
+export { Graders } from './graders/graders';
export {
Images,
type Image,
diff --git src/resources/responses/input-items.ts src/resources/responses/input-items.ts
index c88bb441d..74707f184 100644
--- src/resources/responses/input-items.ts
+++ src/resources/responses/input-items.ts
@@ -10,6 +10,16 @@ import { type CursorPageParams } from '../../pagination';
export class InputItems extends APIResource {
/**
* Returns a list of input items for a given response.
+ *
+ * @example
+ * ```ts
+ * // Automatically fetches more pages as needed.
+ * for await (const responseItem of client.responses.inputItems.list(
+ * 'response_id',
+ * )) {
+ * // ...
+ * }
+ * ```
*/
list(
responseId: string,
diff --git src/resources/responses/responses.ts src/resources/responses/responses.ts
index 771b8daf2..40dd9a6fb 100644
--- src/resources/responses/responses.ts
+++ src/resources/responses/responses.ts
@@ -37,7 +37,13 @@ export type ParsedResponseOutputItem<ParsedT> =
| ResponseFileSearchToolCall
| ResponseFunctionWebSearch
| ResponseComputerToolCall
- | ResponseReasoningItem;
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
export interface ParsedResponse<ParsedT> extends Response {
output: Array<ParsedResponseOutputItem<ParsedT>>;
@@ -61,6 +67,14 @@ export class Responses extends APIResource {
* [web search](https://platform.openai.com/docs/guides/tools-web-search) or
* [file search](https://platform.openai.com/docs/guides/tools-file-search) to use
* your own data as input for the model's response.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.create({
+ * input: 'string',
+ * model: 'gpt-4o',
+ * });
+ * ```
*/
create(body: ResponseCreateParamsNonStreaming, options?: Core.RequestOptions): APIPromise<Response>;
create(
@@ -90,6 +104,13 @@ export class Responses extends APIResource {
/**
* Retrieves a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * const response = await client.responses.retrieve(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
retrieve(
responseId: string,
@@ -110,6 +131,13 @@ export class Responses extends APIResource {
/**
* Deletes a model response with the given ID.
+ *
+ * @example
+ * ```ts
+ * await client.responses.del(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
*/
del(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
return this._client.delete(`/responses/${responseId}`, {
@@ -130,12 +158,33 @@ export class Responses extends APIResource {
/**
* Creates a model response stream
*/
+
stream<Params extends ResponseStreamParams, ParsedT = ExtractParsedContentFromParams<Params>>(
body: Params,
options?: Core.RequestOptions,
): ResponseStream<ParsedT> {
return ResponseStream.createResponse<ParsedT>(this._client, body, options);
}
+
+ /**
+ * Cancels a model response with the given ID. Only responses created with the
+ * `background` parameter set to `true` can be cancelled.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ *
+ * @example
+ * ```ts
+ * await client.responses.cancel(
+ * 'resp_677efb5139a88190b512bc3fef8e535d',
+ * );
+ * ```
+ */
+
+ cancel(responseId: string, options?: Core.RequestOptions): Core.APIPromise<void> {
+ return this._client.post(`/responses/${responseId}/cancel`, {
+ ...options,
+ headers: { Accept: '*/*', ...options?.headers },
+ });
+ }
}
export class ResponseItemsPage extends CursorPage<ResponseItem> {}
@@ -158,7 +207,7 @@ export interface ComputerTool {
/**
* The type of computer environment to control.
*/
- environment: 'mac' | 'windows' | 'ubuntu' | 'browser';
+ environment: 'windows' | 'mac' | 'linux' | 'ubuntu' | 'browser';
/**
* The type of the computer use tool. Always `computer_use_preview`.
@@ -209,9 +258,9 @@ export interface FileSearchTool {
vector_store_ids: Array<string>;
/**
- * A filter to apply based on file attributes.
+ * A filter to apply.
*/
- filters?: Shared.ComparisonFilter | Shared.CompoundFilter;
+ filters?: Shared.ComparisonFilter | Shared.CompoundFilter | nu,ll;
/**
* The maximum number of results to return. This number should be between 1 and 50
@@ -258,12 +307,12 @@ export interface FunctionTool {
/**
* A JSON schema object describing the parameters of the function.
*/
- parameters: Record<string, unknown>;
+ parameters: Record<string, unknown> | null;
/**
* Whether to enforce strict parameter validation. Default `true`.
*/
- strict: boolean;
+ strict: boolean | null;
/**
* The type of the function tool. Always `function`.
@@ -392,6 +441,12 @@ export interface Response {
*/
top_p: number | null;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* An upper bound for the number of tokens that can be generated for a response,
* including visible output tokens and
@@ -437,7 +492,7 @@ export interface Response {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
status?: ResponseStatus;
@@ -496,6 +551,11 @@ export interface ResponseAudioDeltaEvent {
*/
delta: string;
+ /**
+ * A sequence number for this chunk of the stream response.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.delta`.
*/
@@ -506,6 +566,11 @@ export interface ResponseAudioDeltaEvent {
* Emitted when the audio response is complete.
*/
export interface ResponseAudioDoneEvent {
+ /**
+ * The sequence number of the delta.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.done`.
*/
@@ -521,6 +586,11 @@ export interface ResponseAudioTranscriptDeltaEvent {
*/
delta: string;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.transcript.delta`.
*/
@@ -531,6 +601,11 @@ export interface ResponseAudioTranscriptDeltaEvent {
* Emitted when the full audio transcript is completed.
*/
export interface ResponseAudioTranscriptDoneEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.audio.transcript.done`.
*/
@@ -551,6 +626,11 @@ export interface ResponseCodeInterpreterCallCodeDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.code.delta`.
*/
@@ -571,6 +651,11 @@ export interface ResponseCodeInterpreterCallCodeDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.code.done`.
*/
@@ -591,6 +676,11 @@ export interface ResponseCodeInterpreterCallCompletedEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.completed`.
*/
@@ -611,6 +701,11 @@ export interface ResponseCodeInterpreterCallInProgressEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.in_progress`.
*/
@@ -631,6 +726,11 @@ export interface ResponseCodeInterpreterCallInterpretingEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.code_interpreter_call.interpreting`.
*/
@@ -665,6 +765,11 @@ export interface ResponseCodeInterpreterToolCall {
* The type of the code interpreter tool call. Always `code_interpreter_call`.
*/
type: 'code_interpreter_call';
+
+ /**
+ ,* The ID of the container used to run the code.
+ */
+ container_id?: string;
}
export namespace ResponseCodeInterpreterToolCall {
@@ -719,6 +824,11 @@ export interface ResponseCompletedEvent {
*/
response: Response;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.completed`.
*/
@@ -1101,6 +1211,11 @@ export interface ResponseContentPartAddedEvent {
*/
part: ResponseOutputText | ResponseOutputRefusal;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.content_part.added`.
*/
@@ -1131,6 +1246,11 @@ export interface ResponseContentPartDoneEvent {
*/
part: ResponseOutputText | ResponseOutputRefusal;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.content_part.done`.
*/
@@ -1146,6 +1266,11 @@ export interface ResponseCreatedEvent {
*/
response: Response;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.created`.
*/
@@ -1204,6 +1329,11 @@ export interface ResponseErrorEvent {
*/
param: string | null;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `error`.
*/
@@ -1219,6 +1349,11 @@ export interface ResponseFailedEvent {
*/
response: Response;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.failed`.
*/
@@ -1239,6 +1374,11 @@ export interface ResponseFileSearchCallCompletedEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.file_search_call.completed`.
*/
@@ -1259,6 +1399,11 @@ export interface ResponseFileSearchCallInProgressEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.file_search_call.in_progress`.
*/
@@ -1279,6 +1424,11 @@ export interface ResponseFileSearchCallSearchingEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.file_search_call.searching`.
*/
@@ -1429,6 +1579,11 @@ export interface ResponseFunctionCallArgumentsDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.function_call_arguments.delta`.
*/
@@ -1454,6 +1609,11 @@ export interface ResponseFunctionCallArgumentsDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
type: 'response.function_call_arguments.done';
}
@@ -1557,6 +1717,119 @@ export interface ResponseFunctionWebSearch {
type: 'web_search_call';
}
+/**
+ * Emitted when an image generation tool call has completed and the final image is
+ * available.
+ */
+export interface ResponseImageGenCallCompletedEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.completed'.
+ */
+ type: 'response.image_generation_call.completed';
+}
+
+/**
+ * Emitted when an image generation tool call is actively generating an image
+ * (intermediate state).
+ */
+export interface ResponseImageGenCallGeneratingEvent {
+ /**
+ * The unique identifier of the image generation item b,eing processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.generating'.
+ */
+ type: 'response.image_generation_call.generating';
+}
+
+/**
+ * Emitted when an image generation tool call is in progress.
+ */
+export interface ResponseImageGenCallInProgressEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.in_progress'.
+ */
+ type: 'response.image_generation_call.in_progress';
+}
+
+/**
+ * Emitted when a partial image is available during image generation streaming.
+ */
+export interface ResponseImageGenCallPartialImageEvent {
+ /**
+ * The unique identifier of the image generation item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * Base64-encoded partial image data, suitable for rendering as an image.
+ */
+ partial_image_b64: string;
+
+ /**
+ * 0-based index for the partial image (backend is 1-based, but this is 0-based for
+ * the user).
+ */
+ partial_image_index: number;
+
+ /**
+ * The sequence number of the image generation item being processed.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.image_generation_call.partial_image'.
+ */
+ type: 'response.image_generation_call.partial_image';
+}
+
/**
* Emitted when the response is in progress.
*/
@@ -1566,6 +1839,11 @@ export interface ResponseInProgressEvent {
*/
response: Response;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.in_progress`.
*/
@@ -1581,11 +1859,17 @@ export interface ResponseInProgressEvent {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
export type ResponseIncludable =
| 'file_search_call.results'
| 'message.input_image.image_url'
- | 'computer_call_output.output.image_url';
+ | 'computer_call_output.output.image_url'
+ | 'reasoning.encrypted_content';
/**
* An event that is emitted when a response finishes as incomplete.
@@ -1596,6 +1880,11 @@ export interface ResponseIncompleteEvent {
*/
response: Response;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.incomplete`.
*/
@@ -1650,7 +1939,7 @@ export interface ResponseInputFile {
/**
* The ID of the file to be sent to the model.
*/
- file_id?: string;
+ file_id?: string | null;
/**
* The name of the file to be sent to the model.
@@ -1667,7 +1956,7 @@ export interface ResponseInputImage {
* The detail level of the image to be sent to the model. One of `high`, `low`, or
* `auto`. Defaults to `auto`.
*/
- detail: 'high' | 'low' | 'auto';
+ detail: 'low' | 'high' | 'auto';
/**
* The type of the input item. Always `input_image`.
@@ -1704,6 +1993,14 @@ export type ResponseInputItem =
| ResponseFunctio,nToolCall
| ResponseInputItem.FunctionCallOutput
| ResponseReasoningItem
+ | ResponseInputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseInputItem.LocalShellCall
+ | ResponseInputItem.LocalShellCallOutput
+ | ResponseInputItem.McpListTools
+ | ResponseInputItem.McpApprovalRequest
+ | ResponseInputItem.McpApprovalResponse
+ | ResponseInputItem.McpCall
| ResponseInputItem.ItemReference;
export namespace ResponseInputItem {
@@ -1758,19 +2055,19 @@ export namespace ResponseInputItem {
/**
* The ID of the computer tool call output.
*/
- id?: string;
+ id?: string | null;
/**
* The safety checks reported by the API that have been acknowledged by the
* developer.
*/
- acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck>;
+ acknowledged_safety_checks?: Array<ComputerCallOutput.AcknowledgedSafetyCheck> | null;
/**
* The status of the message input. One of `in_progress`, `completed`, or
* `incomplete`. Populated when input items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
export namespace ComputerCallOutput {
@@ -1786,12 +2083,12 @@ export namespace ResponseInputItem {
/**
* The type of the pending safety check.
*/
- code: string;
+ code?: string | null;
/**
* Details about the pending safety check.
*/
- message: string;
+ message?: string | null;
}
}
@@ -1818,104 +2115,824 @@ export namespace ResponseInputItem {
* The unique ID of the function tool call output. Populated when this item is
* returned via API.
*/
- id?: string;
+ id?: string | null;
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
*/
- status?: 'in_progress' | 'completed' | 'incomplete';
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
}
/**
- * An internal identifier for an item to reference.
+ * An image generation request made by the model.
*/
- export interface ItemReference {
+ export interface ImageGenerationCall {
/**
- * The ID of the item to reference.
+ * The unique ID of the image generation call.
*/
id: string;
/**
- * The type of item to reference. Always `item_reference`.
+ * The generated image encoded in base64.
*/
- type: 'item_reference';
- }
-}
+ result: string | null;
-/**
- * A list of one or many input items to the model, containing different content
- * types.
- */
-export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
-export interface ResponseInputMessageItem {
- /**
- * The unique ID of the message input.
- */
- id: string;
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
/**
- * A list of one or many input items to the model, containing different content
- * types.
+ * A tool call to run a command on the local shell.
*/
- content: ResponseInputMessageContentList;
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
- /**
- * The role of the message input. One of `user`, `system`, or `developer`.
- */
- role: 'user' | 'system' | 'developer';
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
- /**
- * The status of item. One of `in_progress`, `completed`, or `incomplete`.
- * Populated when items are returned via API.
- */
- status?: 'in_progress' | 'completed' | 'incomplete';
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
- /**
- * The type of the message input. Always set to `message`.
- */
- type?: 'message';
-}
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
-/**
- * A text input to the model.
- */
-export interface ResponseInputText {
- /**
- * The text input to the model.
- */
- text: string;
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
- /**
- * The type of the input item. Always `input_text`.
- */
- type: 'input_text';
-}
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
-/**
- * Content item used to generate a response.
- */
-export type ResponseItem =
- | ResponseInputMessageItem
- | ResponseOutputMessage
- | ResponseFileSearchToolCall
- | ResponseComputerToolCall
- | ResponseComputerToolCallOutputItem
- | ResponseFunctionWebSearch
- | ResponseFunctionToolCallItem
- | ResponseFunctionToolCallOutputItem;
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
-/**
- * An audio output from the model.
- */
-export interface ResponseOutputAudio {
- /**
- * Base64-encoded audio data from the model.
- */
- data: string;
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
- /**
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * The output of a local shell tool call.
+ */
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
+
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
+
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * The unique ID of the approval response
+ */
+ id?: string | null;
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * An internal identifier for an item to reference.
+ */
+ export interface ItemReference {
+ /**
+ * The ID of the item to reference.
+ */
+ id: string;
+
+ /**
+ * The type of item to reference. Always `item_reference`.
+ */
+ type?: 'item_reference' | null;
+ }
+}
+
+/**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+export type ResponseInputMessageContentList = Array<ResponseInputContent>;
+
+export interface ResponseInputMessageItem {
+ /**
+ * The unique ID of the message input.
+ */
+ id: string;
+
+ /**
+ * A list of one or many input items to the model, containing different content
+ * types.
+ */
+ content: ResponseInputMessageContentList;
+
+ /**
+ * The role of the message input. One of `user`, `system`, or `developer`.
+ */
+ role: 'user' | 'system' | 'developer';
+
+ /**
+ * The status of item. One of `in_progress`, `completed`, or `incomplete`.
+ * Populated when items are returned via API.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the message input. Always set to `message`.
+ */
+ type?: 'message';
+}
+
+/**
+ * A text input to the model.
+ */
+export interface ResponseInputText {
+ /**
+ * The text input to the model.
+ */
+ text: string;
+
+ /**
+ * The type of the input item. Always `input_text`.
+ */
+ type: 'input_text';
+}
+
+/**
+ * Content item used to generate a response.
+ */
+export type ResponseItem =
+ | ResponseInputMessageItem
+ | ResponseOutputMessage
+ | ResponseFileSearchToolCall
+ | ResponseComputerToolCall
+ | ResponseComputerToolCallOutputItem
+ | ResponseFunctionWebSearch
+ | ResponseFunctionToolCallItem
+ | ResponseFunctionToolCallOutputItem
+ | ResponseItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseItem.LocalShellCall
+ | ResponseItem.LocalShellCallOutput
+ | ResponseItem.McpListTools
+ | ResponseItem.McpApprovalRequest
+ | ResponseItem.McpApprovalResponse
+ | ResponseItem.McpCall;
+
+export namespace ResponseItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * The output of a local shell tool call.
+ */
+ export interface LocalShellCallOutput {
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the output of the local shell tool call.
+ */
+ output: string;
+
+ /**
+ * The type of the local shell tool call output. Always `local_shell_call_output`.
+ */
+ type: 'local_shell_call_output';
+
+ /**
+ * The status of the item. One of `in_progress`, `completed`, or `incomplete`.
+ */
+ status?: 'in_progress' | 'completed' | 'incomplete' | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+
+ /**
+ * A response to an MCP approval request.
+ */
+ export interface McpApprovalResponse {
+ /**
+ * The unique ID of the approval response
+ */
+ id: string;
+
+ /**
+ * The ID of the approval request being answered.
+ */
+ approval_request_id: string;
+
+ /**
+ * Whether the request was approved.
+ */
+ approve: boolean;
+
+ /**
+ * The type of the item. Always `mcp_approval_response`.
+ */
+ type: 'mcp_approval_response';
+
+ /**
+ * Optional reason for the decision.
+ */
+ reason?: string | null;
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the arguments of an MCP tool
+ * call.
+ */
+export interface ResponseMcpCallArgumentsDeltaEvent {
+ /**
+ * The partial update to the arguments for the MCP tool call.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_delta'.
+ */
+ type: 'response.mcp_call.arguments_delta';
+}
+
+/**
+ * Emitted when the arguments for an MCP tool call are finalized.
+ */
+export interface ResponseMcpCallArgumentsDoneEvent {
+ /**
+ * The finalized arguments for the MCP tool call.
+ */
+ arguments: unknown;
+
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.arguments_done'.
+ */
+ type: 'response.mcp_call.arguments_done';
+}
+
+/**
+ * Emitted when an MCP tool call has completed successfully.
+ */
+export interface ResponseMcpCallCompletedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.completed'.
+ */
+ type: 'response.mcp_call.completed';
+}
+
+/**
+ * Emitted when an MCP tool call has failed.
+ */
+export interface ResponseMcpCallFailedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.failed'.
+ */
+ type: 'response.mcp_call.failed';
+}
+
+/**
+ * Emitted when an MCP tool call is in progress.
+ */
+export interface ResponseMcpCallInProgressEvent {
+ /**
+ * The unique identifier of the MCP tool call item being processed.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_call.in_progress'.
+ */
+ type: 'response.mcp_call.in_progress';
+}
+
+/**
+ * Emitted when the list of available MCP tools has been successfully retrieved.
+ */
+export interface ResponseMcpListToolsCompletedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.completed'.
+ */
+ type: 'response.mcp_list_tools.completed';
+}
+
+/**
+ * Emitted when the attempt to list available MCP tools has failed.
+ */
+export interface ResponseMcpListToolsFailedEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.failed'.
+ */
+ type: 'response.mcp_list_tools.failed';
+}
+
+/**
+ * Emitted when the system is in the process of retrieving the list of available
+ * MCP tools.
+ */
+export interface ResponseMcpListToolsInProgressEvent {
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.mcp_list_tools.in_progress'.
+ */
+ type: 'response.mcp_list_tools.in_progress';
+}
+
+/**
+ * An audio output from the model.
+ */
+export interface ResponseOutputAudio {
+ /**
+ * Base64-encoded audio data from the model.
+ */
+ data: string;
+
+ /**
* The transcript of the audio data from the model.
*/
transcript: string;
@@ -1935,7 +2952,234 @@ export type ResponseOutputItem =
| ResponseFunctionToolCall
| ResponseFunctionWebSearch
| ResponseComputerToolCall
- | ResponseReasoningItem;
+ | ResponseReasoningItem
+ | ResponseOutputItem.ImageGenerationCall
+ | ResponseCodeInterpreterToolCall
+ | ResponseOutputItem.LocalShellCall
+ | ResponseOutputItem.McpCall
+ | ResponseOutputItem.McpListTools
+ | ResponseOutputItem.McpApprovalRequest;
+
+export namespace ResponseOutputItem {
+ /**
+ * An image generation request made by the model.
+ */
+ export interface ImageGenerationCall {
+ /**
+ * The unique ID of the image generation call.
+ */
+ id: string;
+
+ /**
+ * The generated image encoded in base64.
+ */
+ result: string | null;
+
+ /**
+ * The status of the image generation call.
+ */
+ status: 'in_progress' | 'completed' | 'generating' | 'failed';
+
+ /**
+ * The type of the image generation call. Always `image_generation_call`.
+ */
+ type: 'image_generation_call';
+ }
+
+ /**
+ * A tool call to run a command on the local shell.
+ */
+ export interface LocalShellCall {
+ /**
+ * The unique ID of the local shell call.
+ */
+ id: string;
+
+ /**
+ * Execute a shell command on the server.
+ */
+ action: LocalShellCall.Action;
+
+ /**
+ * The unique ID of the local shell tool call generated by the model.
+ */
+ call_id: string;
+
+ /**
+ * The status of the local shell call.
+ */
+ status: 'in_progress' | 'completed' | 'incomplete';
+
+ /**
+ * The type of the local shell call. Always `local_shell_call`.
+ */
+ type: 'local_shell_call';
+ }
+
+ export namespace LocalShellCall {
+ /**
+ * Execute a shell command on the server.
+ */
+ export interface Action {
+ /**
+ * The command to run.
+ */
+ command: Array<string>;
+
+ /**
+ * Environment variables to set for the command.
+ */
+ env: Record<string, string>;
+
+ /**
+ * The type of the local shell action. Always `exec`.
+ */
+ type: 'exec';
+
+ /**
+ * Optional timeout in milliseconds for the command.
+ */
+ timeout_ms?: number | null;
+
+ /**
+ * Optional user to run the command as.
+ */
+ user?: string | null;
+
+ /**
+ * Optional working directory to run the command in.
+ */
+ working_directory?: string | null;
+ }
+ }
+
+ /**
+ * An invocation of a tool on an MCP server.
+ */
+ export interface McpCall {
+ /**
+ * The unique ID of the tool call.
+ */
+ id: string;
+
+ /**
+ * A JSON string of the arguments passed to the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool that was run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server running the tool.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_call`.
+ */
+ type: 'mcp_call';
+
+ /**
+ * The error from the tool call, if any.
+ */
+ error?: string | null;
+
+ /**
+ * The output from the tool call.
+ */
+ output?: string | null;
+ }
+
+ /**
+ * A list of tools available on an MCP server.
+ */
+ export interface McpListTools {
+ /**
+ * The unique ID of the list.
+ */
+ id: string;
+
+ /**
+ * The label of the MCP server.
+ */
+ server_label: string;
+
+ /**
+ * The tools available on the server.
+ */
+ tools: Array<McpListTools.Tool>;
+
+ /**
+ * The type of the item. Always `mcp_list_tools`.
+ */
+ type: 'mcp_list_tools';
+
+ /**
+ * Error message if the server could not list tools.
+ */
+ error?: string | null;
+ }
+
+ export namespace McpListTools {
+ /**
+ * A tool available on an MCP server.
+ */
+ export interface Tool {
+ /**
+ * The JSON schema describing the tool's input.
+ */
+ input_schema: unknown;
+
+ /**
+ * The name of the tool.
+ */
+ name: string;
+
+ /**
+ * Additional annotations about the tool.
+ */
+ annotations?: unknown | null;
+
+ /**
+ * The description of the tool.
+ */
+ description?: string | null;
+ }
+ }
+
+ /**
+ * A request for human approval of a tool invocation.
+ */
+ export interface McpApprovalRequest {
+ /**
+ * The unique ID of the approval request.
+ */
+ id: string;
+
+ /**
+ * A JSON string of arguments for the tool.
+ */
+ arguments: string;
+
+ /**
+ * The name of the tool to run.
+ */
+ name: string;
+
+ /**
+ * The label of the MCP server making the request.
+ */
+ server_label: string;
+
+ /**
+ * The type of the item. Always `mcp_approval_request`.
+ */
+ type: 'mcp_approval_request';
+ }
+}
/**
* Emitted when a new output item is added.
@@ -1951,6 +3195,11 @@ export interface ResponseOutputItemAddedEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_item.added`.
*/
@@ -1971,6 +3220,11 @@ export interface ResponseOutputItemDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_item.done`.
*/
@@ -2075,51 +3329,183 @@ export namespace ResponseOutputText {
*/
end_index: number;
- /**
- * The index of the first character of the URL citation in the message.
- */
- start_index: number;
+ /**
+ * The index of the first character of the URL citation in the message.
+ */
+ start_index: number;
+
+ /**
+ * The title of the web resource.
+ */
+ title: string;
+
+ /**
+ * The type of the URL citation. Always `url_citation`.
+ */
+ type: 'url_citation';
+
+ /**
+ * The URL of the web resource.
+ */
+ url: string;
+ }
+
+ /**
+ * A path to a file.
+ */
+ export interface FilePath {
+ /**
+ * The ID of the file.
+ */
+ file_id: string;
+
+ /**
+ * The index of the file in the list of files.
+ */
+ index: number;
+
+ /**
+ * The type of the file path. Always `file_path`.
+ */
+ type: 'file_path';
+ }
+}
+
+/**
+ * Emitted when an annotation is added to output text content.
+ */
+export interface ResponseOutputTextAnnotationAddedEvent {
+ /**
+ * The annotation object being added. (See annotation schema for details.)
+ */
+ annotation: unknown;
+
+ /**
+ * The index of the annotation within the content part.
+ */
+ annotation_index: number;
+
+ /**
+ * The index of the content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The unique identifier of the item to which the annotation is being added.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.output_text_annotation.added'.
+ */
+ type: 'response.output_text_annotation.added';
+}
+
+/**
+ * Emitted when a response is queued and waiting to be processed.
+ */
+export interface ResponseQueuedEvent {
+ /**
+ * The full response object that is queued.
+ */
+ response: Response;
+
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.queued'.
+ */
+ type: 'response.queued';
+}
+
+/**
+ * Emitted when there is a delta (partial update) to the reasoning content.
+ */
+export interface ResponseReasoningDeltaEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
+
+ /**
+ * The partial update to the reasoning content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which reasoning is being updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning.delta'.
+ */
+ type: 'response.reasoning.delta';
+}
- /**
- * The title of the web resource.
- */
- title: string;
+/**
+ * Emitted when the reasoning content is finalized for an item.
+ */
+export interface ResponseReasoningDoneEvent {
+ /**
+ * The index of the reasoning content part within the output item.
+ */
+ content_index: number;
- /**
- * The type of the URL citation. Always `url_citation`.
- */
- type: 'url_citation';
+ /**
+ * The unique identifier of the item for which reasoning is finalized.
+ */
+ item_id: string;
- /**
- * The URL of the web resource.
- */
- url: string;
- }
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
/**
- * A path to a file.
+ * The sequence number of this event.
*/
- export interface FilePath {
- /**
- * The ID of the file.
- */
- file_id: string;
+ sequence_number: number;
- /**
- * The index of the file in the list of files.
- */
- index: number;
+ /**
+ * The finalized reasoning text.
+ */
+ text: string;
- /**
- * The type of the file path. Always `file_path`.
- */
- type: 'file_path';
- }
+ /**
+ * The type of the event. Always 'response.reasoning.done'.
+ */
+ type: 'response.reasoning.done';
}
/**
* A description of the chain of thought used by a reasoning model while generating
- * a response.
+ * a response. Be sure to include these items in your `input` to the Responses API
+ * for subsequent turns of a conversation if you are manually
+ * [managing context](https://platform.openai.com/docs/guides/conversation-state).
*/
export interface ResponseReasoningItem {
/**
@@ -2137,6 +3523,12 @@ export interface ResponseReasoningItem {
*/
type: 'reasoning';
+ /**
+ * The encrypted content of the reasoning item - populated when a response is
+ * generated with `reasoning.encrypted_content` in the `include` parameter.
+ */
+ encrypted_content?: string | null;
+
/**
* The status of the item. One of `in_progress`, `completed`, or `incomplete`.
* Populated when items are returned via API.
@@ -2158,6 +3550,77 @@ export namespace ResponseReasoningItem {
}
}
+/**
+ * Emitted when there is a delta (partial update) to the reasoning summary content.
+ */
+export interface ResponseReasoningSummaryDeltaEvent {
+ /**
+ * The partial update to the reasoning summary content.
+ */
+ delta: unknown;
+
+ /**
+ * The unique identifier of the item for which the reasoning summary is being
+ * updated.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.delta'.
+ */
+ type: 'response.reasoning_summary.delta';
+}
+
+/**
+ * Emitted when the reasoning summary content is finalized for an item.
+ */
+export interface ResponseReasoningSummaryDoneEvent {
+ /**
+ * The unique identifier of the item for which the reasoning summary is finalized.
+ */
+ item_id: string;
+
+ /**
+ * The index of the output item in the response's output array.
+ */
+ output_index: number;
+
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
+ /**
+ * The index of the summary part within the output item.
+ */
+ summary_index: number;
+
+ /**
+ * The finalized reasoning summary text.
+ */
+ text: string;
+
+ /**
+ * The type of the event. Always 'response.reasoning_summary.done'.
+ */
+ type: 'response.reasoning_summary.done';
+}
+
/**
* Emitted when a new reasoning summary part is added.
*/
@@ -2177,6 +3640,11 @@ export interface ResponseReasoningSummaryPartAddedEvent {
*/
part: ResponseReasoningSummaryPartAddedEvent.Part;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2224,6 +3692,11 @@ export interface ResponseReasoningSummaryPartDoneEvent {
*/
part: ResponseReasoningSummaryPartDoneEvent.Part;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2271,6 +3744,11 @@ export interface ResponseReasoningSummaryTextDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2296,6 +3774,11 @@ export interface ResponseReasoningSummaryTextDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The index of the summary part within the reasoning summary.
*/
@@ -2336,6 +3819,11 @@ export interface ResponseRefusalDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.refusal.delta`.
*/
@@ -2366,6 +3854,11 @@ export interface ResponseRefusalDoneEvent {
*/
refusal: string;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.refusal.done`.
*/
@@ -2374,9 +3867,9 @@ export interface ResponseRefusalDoneEvent {
/**
* The status of the response generation. One of `completed`, `failed`,
- * `in_progress`, or `incomplete`.
+ * `in_progress`, `cancelled`, `queued`, or `incomplete`.
*/
-export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'incomplete';
+export type ResponseStatus = 'completed' | 'failed' | 'in_progress' | 'cancelled' | 'queued' | 'incomplete';
/**
* Emitted when there is a partial audio response.
@@ -2417,7 +3910,25 @@ export type ResponseStreamEvent =
| ResponseTextDoneEvent
| ResponseWebSearchCallCompletedEvent
| ResponseWebSearchCallInProgressEvent
- | ResponseWebSearchCallSearchingEvent;
+ | ResponseWebSearchCallSearchingEvent
+ | ResponseImageGenCallCompletedEvent
+ | ResponseImageGenCallGeneratingEvent
+ | ResponseImageGenCallInProgressEvent
+ | ResponseImageGenCallPartialImageEvent
+ | ResponseMcpCallArgumentsDeltaEvent
+ | ResponseMcpCallArgumentsDoneEvent
+ | ResponseMcpCallCompletedEvent
+ | ResponseMcpCallFailedEvent
+ | ResponseMcpCallInProgressEvent
+ | ResponseMcpListToolsCompletedEvent
+ | ResponseMcpListToolsFailedEvent
+ | ResponseMcpListToolsInProgressEvent
+ | ResponseOutputTextAnnotationAddedEvent
+ | ResponseQueuedEvent
+ | ResponseReasoningDeltaEvent
+ | ResponseReasoningDoneEvent
+ | ResponseReasoningSummaryDeltaEvent
+ | ResponseReasoningSummaryDoneEvent;
/**
* Emitted when a text annotation is added.
@@ -2451,6 +3962,11 @@ export interface ResponseTextAnnotationDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number of this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_text.annotation.added`.
*/
@@ -2579,6 +4095,11 @@ export interface ResponseTextDeltaEvent {
*/
output_index: number;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The type of the event. Always `response.output_text.delta`.
*/
@@ -2604,6 +4125,11 @@ export interface ResponseTextDoneEvent {
*/
output_index: number;
+ /**
+ * The sequence number for this event.
+ */
+ sequence_number: number;
+
/**
* The text content that is finalized.
*/
@@ -2730,11 +4256,231 @@ export interface ResponseWebSearchCallSearchingEvent {
}
/**
- * A tool that searches for relevant content from uploaded files. Learn more about
- * the
- * [file search tool](https://platform.openai.com/docs/guides/tools-file-search).
+ * A tool that can be used to generate a response.
*/
-export type Tool = FileSearchTool | FunctionTool | ComputerTool | WebSearchTool;
+export type Tool =
+ | FunctionTool
+ | FileSearchTool
+ | WebSearchTool
+ | ComputerTool
+ | Tool.Mcp
+ | Tool.CodeInterpreter
+ | Tool.ImageGeneration
+ | Tool.LocalShell;
+
+export namespace Tool {
+ /**
+ * Give the model access to additional tools via remote Model Context Protocol
+ * (MCP) servers.
+ * [Learn more about MCP](https://platform.openai.com/docs/guides/tools-remote-mcp).
+ */
+ export interface Mcp {
+ /**
+ * A label for this MCP server, used to identify it in tool calls.
+ */
+ server_label: string;
+
+ /**
+ * The URL for the MCP server.
+ */
+ server_url: string;
+
+ /**
+ * The type of the MCP tool. Always `mcp`.
+ */
+ type: 'mcp';
+
+ /**
+ * List of allowed tool names or a filter object.
+ */
+ allowed_tools?: Array<string> | Mcp.McpAllowedToolsFilter | null;
+
+ /**
+ * Optional HTTP headers to send to the MCP server. Use for authentication or other
+ * purposes.
+ */
+ headers?: Record<string, string> | null;
+
+ /**
+ * Specify which of the MCP server's tools require approval.
+ */
+ require_approval?: Mcp.McpToolApprovalFilter | 'always' | 'never' | null;
+ }
+
+ export namespace Mcp {
+ /**
+ * A filter object to specify which tools are allowed.
+ */
+ export interface McpAllowedToolsFilter {
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export interface McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ always?: McpToolApprovalFilter.Always;
+
+ /**
+ * A list of tools that never require approval.
+ */
+ never?: McpToolApprovalFilter.Never;
+
+ /**
+ * List of allowed tool names.
+ */
+ tool_names?: Array<string>;
+ }
+
+ export namespace McpToolApprovalFilter {
+ /**
+ * A list of tools that always require approval.
+ */
+ export interface Always {
+ /**
+ * List of tools that require approval.
+ */
+ tool_names?: Array<string>;
+ }
+
+ /**
+ * A list of tools that never require approval.
+ */
+ export interface Never {
+ /**
+ * List of tools that do not require approval.
+ */
+ tool_names?: Array<string>;
+ }
+ }
+ }
+
+ /**
+ * A tool that runs Python code to help generate a response to a prompt.
+ */
+ export interface CodeInterpreter {
+ /**
+ * The code interpreter container. Can be a container ID or an object that
+ * specifies uploaded file IDs to make available to your code.
+ */
+ container: string | CodeInterpreter.CodeInterpreterToolAuto;
+
+ /**
+ * The type of the code interpreter tool. Always `code_interpreter`.
+ */
+ type: 'code_interpreter';
+ }
+
+ export namespace CodeInterpreter {
+ /**
+ * Configuration for a code interpreter container. Optionally specify the IDs of
+ * the files to run the code on.
+ */
+ export interface CodeInterpreterToolAuto {
+ /**
+ * Always `auto`.
+ */
+ type: 'auto';
+
+ /**
+ * An optional list of uploaded files to make available to your code.
+ */
+ file_ids?: Array<string>;
+ }
+ }
+
+ /**
+ * A tool that generates images using a model like `gpt-image-1`.
+ */
+ export interface ImageGeneration {
+ /**
+ * The type of the image generation tool. Always `image_generation`.
+ */
+ type: 'image_generation';
+
+ /**
+ * Background type for the generated image. One of `transparent`, `opaque`, or
+ * `auto`. Default: `auto`.
+ */
+ background?: 'transparent' | 'opaque' | 'auto';
+
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ input_image_mask?: ImageGeneration.InputImageMask;
+
+ /**
+ * The image generation model to use. Default: `gpt-image-1`.
+ */
+ model?: 'gpt-image-1';
+
+ /**
+ * Moderation level for the generated image. Default: `auto`.
+ */
+ moderation?: 'auto' | 'low';
+
+ /**
+ * Compression level for the output image. Default: 100.
+ */
+ output_compression?: number;
+
+ /**
+ * The output format of the generated image. One of `png`, `webp`, or `jpeg`.
+ * Default: `png`.
+ */
+ output_format?: 'png' | 'webp' | 'jpeg';
+
+ /**
+ * Number of partial images to generate in streaming mode, from 0 (default value)
+ * to 3.
+ */
+ partial_images?: number;
+
+ /**
+ * The quality of the generated image. One of `low`, `medium`, `high`, or `auto`.
+ * Default: `auto`.
+ */
+ quality?: 'low' | 'medium' | 'high' | 'auto';
+
+ /**
+ * The size of the generated image. One of `1024x1024`, `1024x1536`, `1536x1024`,
+ * or `auto`. Default: `auto`.
+ */
+ size?: '1024x1024' | '1024x1536' | '1536x1024' | 'auto';
+ }
+
+ export namespace ImageGeneration {
+ /**
+ * Optional mask for inpainting. Contains `image_url` (string, optional) and
+ * `file_id` (string, optional).
+ */
+ export interface InputImageMask {
+ /**
+ * File ID for the mask image.
+ */
+ file_id?: string;
+
+ /**
+ * Base64-encoded mask image.
+ */
+ image_url?: string;
+ }
+ }
+
+ /**
+ * A tool that allows the model to execute shell commands in a local environment.
+ */
+ export interface LocalShell {
+ /**
+ * The type of the local shell tool. Always `local_shell`.
+ */
+ type: 'local_shell';
+ }
+}
/**
* Use this option to force the model to call a specific function.
@@ -2777,8 +4523,18 @@ export interface ToolChoiceTypes {
* - `file_search`
* - `web_search_preview`
* - `computer_use_preview`
- */
- type: 'file_search' | 'web_search_preview' | 'computer_use_preview' | 'web_search_preview_2025_03_11';
+ * - `code_interpreter`
+ * - `mcp`
+ * - `image_generation`
+ */
+ type:
+ | 'file_search'
+ | 'web_search_preview'
+ | 'computer_use_preview'
+ | 'web_search_preview_2025_03_11'
+ | 'image_generation'
+ | 'code_interpreter'
+ | 'mcp';
}
/**
@@ -2788,10 +4544,8 @@ export interface ToolChoiceTypes {
*/
export interface WebSearchTool {
/**
- * The type of the web search tool. One of:
- *
- * - `web_search_preview`
- * - `web_search_preview_2025_03_11`
+ * The type of the web search tool. One of `web_search_preview` or
+ * `web_search_preview_2025_03_11`.
*/
type: 'web_search_preview' | 'web_search_preview_2025_03_11';
@@ -2801,10 +4555,16 @@ export interface WebSearchTool {
*/
search_context_size?: 'low' | 'medium' | 'high';
+ /**
+ * The user's location.
+ */
user_location?: WebSearchTool.UserLocation | null;
}
export namespace WebSearchTool {
+ /**
+ * The user's location.
+ */
export interface UserLocation {
/**
* The type of location approximation. Always `approximate`.
@@ -2814,24 +4574,24 @@ export namespace WebSearchTool {
/**
* Free text input for the city of the user, e.g. `San Francisco`.
*/
- city?: string;
+ city?: string | null;
/**
* The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of
* the user, e.g. `US`.
*/
- country?: string;
+ country?: string | null;
/**
* Free text input for the region of the user, e.g. `California`.
*/
- region?: string;
+ region?: string | null;
/**
* The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the
* user, e.g. `America/Los_Angeles`.
*/
- timezone?: string;
+ timezone?: string | null;
}
}
@@ -2860,6 +4620,12 @@ export interface ResponseCreateParamsBase {
*/
model: Shared.ResponsesModel;
+ /**
+ * Whether to run the model response in the background.
+ * [Learn more](https://platform.openai.com/docs/guides/background).
+ */
+ background?: boolean | null;
+
/**
* Specify additional output data to include in the model response. Currently
* supported values are:
@@ -2869,6 +4635,11 @@ export interface ResponseCreateParamsBase {
* - `message.input_image.image_url`: Include image urls from the input message.
* - `computer_call_output.output.image_url`: Include image urls from the computer
* call output.
+ * - `reasoning.encrypted_content`: Includes an encrypted version of reasoning
+ * tokens in reasoning item outputs. This enables reasoning items to be used in
+ * multi-turn conversations when using the Responses API statelessly (like when
+ * the `store` parameter is set to `false`, or when an organization is enrolled
+ * in the zero data retention program).
*/
include?: Array<ResponseIncludable> | null;
@@ -3104,6 +4875,10 @@ export declare namespace Responses {
type ResponseFunctionToolCallItem as ResponseFunctionToolCallItem,
type ResponseFunctionToolCallOutputItem as ResponseFunctionToolCallOutputItem,
type ResponseFunctionWebSearch as ResponseFunctionWebSearch,
+ type ResponseImageGenCallCompletedEvent as ResponseImageGenCallCompletedEvent,
+ type ResponseImageGenCallGeneratingEvent as ResponseImageGenCallGeneratingEvent,
+ type ResponseImageGenCallInProgressEvent as ResponseImageGenCallInProgressEvent,
+ type ResponseImageGenCallPartialImageEvent as ResponseImageGenCallPartialImageEvent,
type ResponseInProgressEvent as ResponseInProgressEvent,
type ResponseIncludable as ResponseIncludable,
type ResponseIncompleteEvent as ResponseIncompleteEvent,
@@ -3117,6 +4892,14 @@ export declare namespace Responses {
type ResponseInputMessageItem as ResponseInputMessageItem,
type ResponseInputText as ResponseInputText,
type ResponseItem as ResponseItem,
+ type ResponseMcpCallArgumentsDeltaEvent as ResponseMcpCallArgumentsDeltaEvent,
+ type ResponseMcpCallArgumentsDoneEvent as ResponseMcpCallArgumentsDoneEvent,
+ type ResponseMcpCallCompletedEvent as ResponseMcpCallCompletedEvent,
+ type ResponseMcpCallFailedEvent as ResponseMcpCallFailedEvent,
+ type ResponseMcpCallInProgressEvent as ResponseMcpCallInProgressEvent,
+ type ResponseMcpListToolsCompletedEvent as ResponseMcpListToolsCompletedEvent,
+ type ResponseMcpListToolsFailedEvent as ResponseMcpListToolsFailedEvent,
+ type ResponseMcpListToolsInProgressEvent as ResponseMcpListToolsInProgressEvent,
type ResponseOutputAudio as ResponseOutputAudio,
type ResponseOutputItem as ResponseOutputItem,
type ResponseOutputItemAddedEvent as ResponseOutputItemAddedEvent,
@@ -3124,7 +4907,13 @@ export declare namespace Responses {
type ResponseOutputMessage as ResponseOutputMessage,
type ResponseOutputRefusal as ResponseOutputRefusal,
type ResponseOutputText as ResponseOutputText,
+ type ResponseOutputTextAnnotationAddedEvent as ResponseOutputTextAnnotationAddedEvent,
+ type ResponseQueuedEvent as ResponseQueuedEvent,
+ type ResponseReasoningDeltaEvent as ResponseReasoningDeltaEvent,
+ type ResponseReasoningDoneEvent as ResponseReasoningDoneEvent,
type ResponseReasoningItem as ResponseReasoningItem,
+ type ResponseReasoningSummaryDeltaEvent as ResponseReasoningSummaryDeltaEvent,
+ type ResponseReasoningSummaryDoneEvent as ResponseReasoningSummaryDoneEvent,
type ResponseReasoningSummaryPartAddedEvent as ResponseReasoningSummaryPartAddedEvent,
type ResponseReasoningSummaryPartDoneEvent as ResponseReasoningSummaryPartDoneEvent,
type ResponseReasoningSummaryTextDeltaEvent as ResponseReasoningSummaryTextDeltaEvent,
diff --git src/resources/shared.ts src/resources/shared.ts
index 1c0006b18..adea184fd 100644
--- src/resources/shared.ts
+++ src/resources/shared.ts
@@ -41,6 +41,7 @@ export type ChatModel =
| 'gpt-4o-search-preview-2025-03-11'
| 'gpt-4o-mini-search-preview-2025-03-11'
| 'chatgpt-4o-latest'
+ | 'codex-mini-latest'
| 'gpt-4o-mini'
| 'gpt-4o-mini-2024-07-18'
| 'gpt-4-turbo'
diff --git src/version.ts src/version.ts
index 1674d74fe..180cea6da 100644
--- src/version.ts
+++ src/version.ts
@@ -1 +1 @@
-export const VERSION = '4.96.2'; // x-release-please-version
+export const VERSION = '4.102.0'; // x-release-please-version
diff --git tests/api-resources/audio/transcriptions.test.ts tests/api-resources/audio/transcriptions.test.ts
index 2297677b4..ad76808d0 100644
--- tests/api-resources/audio/transcriptions.test.ts
+++ tests/api-resources/audio/transcriptions.test.ts
@@ -27,6 +27,7 @@ describe('resource transcriptions', () => {
const response = await client.audio.transcriptions.create({
file: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'gpt-4o-transcribe',
+ chunking_strategy: 'auto',
include: ['logprobs'],
language: 'language',
prompt: 'prompt',
diff --git a/tests/api-resources/containers/containers.test.ts b/tests/api-resources/containers/containers.test.ts
new file mode 100644
index 000000000..11c6f166c
--- /dev/null
+++ tests/api-resources/containers/containers.test.ts
@@ -0,0 +1,94 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource containers', () => {
+ test('create: only required params', async () => {
+ const responsePromise = client.containers.create({ name: 'name' });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('create: required and optional params', async () => {
+ const response = await client.containers.create({
+ name: 'name',
+ expires_after: { anchor: 'last_active_at', minutes: 0 },
+ file_ids: ['string'],
+ });
+ });
+
+ test('retrieve', async () => {
+ const responsePromise = client.containers.retrieve('container_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.retrieve('container_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('list', async () => {
+ const responsePromise = client.containers.list();
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('list: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(client.containers.list({ path: '/_stainless_unknown_path' })).rejects.toThrow(
+ OpenAI.NotFoundError,
+ );
+ });
+
+ test('list: request options and params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.list(
+ { after: 'after', limit: 0, order: 'asc' },
+ { path: '/_stainless_unknown_path' },
+ ),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('del', async () => {
+ const responsePromise = client.containers.del('container_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('del: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(client.containers.del('container_id', { path: '/_stainless_unknown_path' })).rejects.toThrow(
+ OpenAI.NotFoundError,
+ );
+ });
+});
diff --git a/tests/api-resources/containers/files/content.test.ts b/tests/api-resources/containers/files/content.test.ts
new file mode 100644
index 000000000..9215c001a
--- /dev/null
+++ tests/api-resources/containers/files/content.test.ts
@@ -0,0 +1,31 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource content', () => {
+ test('retrieve', async () => {
+ const responsePromise = client.containers.files.content.retrieve('container_id', 'file_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.content.retrieve('container_id', 'file_id', {
+ path: '/_stainless_unknown_path',
+ }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+});
diff --git a/tests/api-resources/containers/files/files.test.ts b/tests/api-resources/containers/files/files.test.ts
new file mode 100644
index 000000000..79f47e9ad
--- /dev/null
+++ tests/api-resources/containers/files/files.test.ts
@@ -0,0 +1,87 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource files', () => {
+ test('create', async () => {
+ const responsePromise = client.containers.files.create('container_id', {});
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve', async () => {
+ const responsePromise = client.containers.files.retrieve('container_id', 'file_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('retrieve: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.retrieve('container_id', 'file_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('list', async () => {
+ const responsePromise = client.containers.files.list('container_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('list: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.list('container_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('list: request options and params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.list(
+ 'container_id',
+ { after: 'after', limit: 0, order: 'asc' },
+ { path: '/_stainless_unknown_path' },
+ ),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('del', async () => {
+ const responsePromise = client.containers.files.del('container_id', 'file_id');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('del: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.containers.files.del('container_id', 'file_id', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+});
diff --git a/tests/api-resources/fine-tuning/alpha/graders.test.ts b/tests/api-resources/fine-tuning/alpha/graders.test.ts
new file mode 100644
index 000000000..8e47a4c42
--- /dev/null
+++ tests/api-resources/fine-tuning/alpha/graders.test.ts
@@ -0,0 +1,53 @@
+// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.
+
+import OpenAI from 'openai';
+import { Response } from 'node-fetch';
+
+const client = new OpenAI({
+ apiKey: 'My API Key',
+ baseURL: process.env['TEST_API_BASE_URL'] ?? 'http://127.0.0.1:4010',
+});
+
+describe('resource graders', () => {
+ test('run: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('run: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.run({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ model_sample: 'model_sample',
+ reference_answer: 'string',
+ });
+ });
+
+ test('validate: only required params', async () => {
+ const responsePromise = client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('validate: required and optional params', async () => {
+ const response = await client.fineTuning.alpha.graders.validate({
+ grader: { input: 'input', name: 'name', operation: 'eq', reference: 'reference', type: 'string_check' },
+ });
+ });
+});
diff --git tests/api-resources/fine-tuning/jobs/jobs.test.ts tests/api-resources/fine-tuning/jobs/jobs.test.ts
index 4de83a8b7..fe8c9efee 100644
--- tests/api-resources/fine-tuning/jobs/jobs.test.ts
+++ tests/api-resources/fine-tuning/jobs/jobs.test.ts
@@ -35,6 +35,7 @@ describe('resource jobs', () => {
},
],
method: {
+ type: 'supervised',
dpo: {
hyperparameters: {
batch_size: 'auto',
@@ -43,10 +44,27 @@ describe('resource jobs', () => {
n_epochs: 'auto',
},
},
+ reinforcement: {
+ grader: {
+ input: 'input',
+ name: 'name',
+ operation: 'eq',
+ reference: 'reference',
+ type: 'string_check',
+ },
+ hyperparameters: {
+ batch_size: 'auto',
+ compute_multiplier: 'auto',
+ eval_interval: 'auto',
+ eval_samples: 'auto',
+ learning_rate_multiplier: 'auto',
+ n_epochs: 'auto',
+ reasoning_effort: 'default',
+ },
+ },
supervised: {
hyperparameters: { batch_size: 'auto', learning_rate_multiplier: 'auto', n_epochs: 'auto' },
},
- type: 'supervised',
},
seed: 42,
suffix: 'x',
@@ -143,4 +161,40 @@ describe('resource jobs', () => {
),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('pause', async () => {
+ const responsePromise = client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndR,esponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('pause: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.pause('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
+
+ test('resume', async () => {
+ const responsePromise = client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('resume: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.fineTuning.jobs.resume('ft-AF1WoRqd3aJAHsqc9NY7iL8F', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
diff --git tests/api-resources/images.test.ts tests/api-resources/images.test.ts
index e9b460254..04fca0a2a 100644
--- tests/api-resources/images.test.ts
+++ tests/api-resources/images.test.ts
@@ -51,6 +51,7 @@ describe('resource images', () => {
const response = await client.images.edit({
image: await toFile(Buffer.from('# my file contents'), 'README.md'),
prompt: 'A cute baby sea otter wearing a beret',
+ background: 'transparent',
mask: await toFile(Buffer.from('# my file contents'), 'README.md'),
model: 'string',
n: 1,
diff --git tests/api-resources/responses/responses.test.ts tests/api-resources/responses/responses.test.ts
index cf7e9cf3c..f8e653c52 100644
--- tests/api-resources/responses/responses.test.ts
+++ tests/api-resources/responses/responses.test.ts
@@ -24,6 +24,7 @@ describe('resource responses', () => {
const response = await client.responses.create({
input: 'string',
model: 'gpt-4o',
+ background: true,
include: ['file_search_call.results'],
instructions: 'instructions',
max_output_tokens: 0,
@@ -39,11 +40,11 @@ describe('resource responses', () => {
tool_choice: 'none',
tools: [
{
- type: 'file_search',
- vector_store_ids: ['string'],
- filters: { key: 'key', type: 'eq', value: 'string' },
- max_num_results: 0,
- ranking_options: { ranker: 'auto', score_threshold: 0 },
+ name: 'name',
+ parameters: { foo: 'bar' },
+ strict: true,
+ type: 'function',
+ description: 'description',
},
],
top_p: 1,
@@ -100,4 +101,22 @@ describe('resource responses', () => {
client.responses.del('resp_677efb5139a88190b512bc3fef8e535d', { path: '/_stainless_unknown_path' }),
).rejects.toThrow(OpenAI.NotFoundError);
});
+
+ test('cancel', async () => {
+ const responsePromise = client.responses.cancel('resp_677efb5139a88190b512bc3fef8e535d');
+ const rawResponse = await responsePromise.asResponse();
+ expect(rawResponse).toBeInstanceOf(Response);
+ const response = await responsePromise;
+ expect(response).not.toBeInstanceOf(Response);
+ const dataAndResponse = await responsePromise.withResponse();
+ expect(dataAndResponse.data).toBe(response);
+ expect(dataAndResponse.response).toBe(rawResponse);
+ });
+
+ test('cancel: request options instead of params are passed correctly', async () => {
+ // ensure the request options are being passed correctly by passing an invalid HTTP method in order to cause an error
+ await expect(
+ client.responses.cancel('resp_677efb5139a88190b512bc3fef8e535d', { path: '/_stainless_unknown_path' }),
+ ).rejects.toThrow(OpenAI.NotFoundError);
+ });
});
Description
This PR upgrades Node.js version from 18 to 20 in CI workflows and introduces several significant API updates to the OpenAI Node.js SDK. The PR includes new endpoints, models, and features such as Containers, reinforcement fine-tuning, an improved Graders system, and enhancements to Responses API. The version is bumped from 4.96.2 to 4.102.0, incorporating multiple releases with various feature additions and bug fixes.
Changes
Changes
CI/Workflow Changes
- Updated Node.js version from 18 to 20 in all GitHub workflow files (
ci.yml,publish-jsr.yml,publish-npm.yml)
Version Updates
- Bumped version from 4.96.2 to 4.102.0 in
.release-please-manifest.json,jsr.json, andpackage.json - Updated stats and configuration in
.stats.ymlshowing an increase from 97 to 111 configured endpoints
New Features and Endpoints
- Containers API: Added new Container management system for file management and storage
- Reinforcement Fine-tuning: Added support for reinforcement learning-based fine-tuning alongside existing supervised and DPO methods
- Graders System: Added structured graders system for evaluating model outputs
- Response Enhancements: Added support for background responses, cancellation, and new output types including image generation
- MCP Integration: Added Model Context Protocol (MCP) support for remote tool execution
Documentation Updates
- Added JSDoc examples to numerous API methods
- Fixed typos in README.md and SECURITY.md
- Added comprehensive sequence diagrams in the API documentation
- Updated documentation for deprecated parameters and methods
Code Improvements
- Reorganized grader models into a dedicated module
- Added more thorough typing for response types and events
- Fixed typos ("procesing" → "processing") in example files
- Added support for new tools including image generation, code interpreter, and local shell access
sequenceDiagram
participant Client
participant OpenAI as OpenAI SDK
participant API as OpenAI API
participant Containers as Container Service
participant FineTuning as Fine-tuning Service
participant Responses as Responses Service
Client->>OpenAI: Initialize with API key
%% Container interaction
Client->>OpenAI: containers.create({name: "myContainer"})
OpenAI->>API: POST /containers
API->>Containers: Create container
Containers-->>API: Return container data
API-->>OpenAI: Container object
OpenAI-->>Client: ContainerCreateResponse
%% Upload file to container
Client->>OpenAI: containers.files.create(containerId, {file})
OpenAI->>API: POST /containers/{containerId}/files
API->>Containers: Store file in container
Containers-->>API: File data
API-->>OpenAI: File object
OpenAI-->>Client: FileCreateResponse
%% Fine-tuning with reinforcement method
Client->>OpenAI: fineTuning.jobs.create({model, training_file, method})
OpenAI->>API: POST /fine_tuning/jobs
API->>FineTuning: Start reinforcement fine-tuning job
FineTuning-->>API: Job data
API-->>OpenAI: FineTuningJob object
OpenAI-->>Client: FineTuningJob
%% Create response with background processing
Client->>OpenAI: responses.create({input, model, background: true})
OpenAI->>API: POST /responses
API->>Responses: Queue response generation
Responses-->>API: Response with queued status
API-->>OpenAI: Response object
OpenAI-->>Client: Response
%% Cancel background response
Client->>OpenAI: responses.cancel(responseId)
OpenAI->>API: POST /responses/{responseId}/cancel
API->>Responses: Cancel response generation
Responses-->>API: Success
API-->>OpenAI: Success response
OpenAI-->>Client: void