## Get eval runs **get** `/evals/{eval_id}/runs` Get a list of runs for an evaluation. ### Path Parameters - `eval_id: string` ### Query Parameters - `after: optional string` Identifier for the last run from the previous pagination request. - `limit: optional number` Number of runs to retrieve. - `order: optional "asc" or "desc"` Sort order for runs by timestamp. Use `asc` for ascending order or `desc` for descending order. Defaults to `asc`. - `"asc"` - `"desc"` - `status: optional "queued" or "in_progress" or "completed" or 2 more` Filter runs by status. One of `queued` | `in_progress` | `failed` | `completed` | `canceled`. - `"queued"` - `"in_progress"` - `"completed"` - `"canceled"` - `"failed"` ### Returns - `data: array of object { id, created_at, data_source, 11 more }` An array of eval run objects. - `id: string` Unique identifier for the evaluation run. - `created_at: number` Unix timestamp (in seconds) when the evaluation run was created. - `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }` Information about the run's data source. - `CreateEvalJSONLRunDataSource object { source, type }` A JsonlRunDataSource object with that specifies a JSONL file that matches the eval - `source: EvalJSONLFileContentSource { content, type } or EvalJSONLFileIDSource { id, type }` Determines what populates the `item` namespace in the data source. - `EvalJSONLFileContentSource object { content, type }` - `content: array of object { item, sample }` The content of the jsonl file. - `item: map[unknown]` - `sample: optional map[unknown]` - `type: "file_content"` The type of jsonl source. Always `file_content`. - `"file_content"` - `EvalJSONLFileIDSource object { id, type }` - `id: string` The identifier of the file. - `type: "file_id"` The type of jsonl source. Always `file_id`. - `"file_id"` - `type: "jsonl"` The type of data source. Always `jsonl`. - `"jsonl"` - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }` A CompletionsRunDataSource object describing a model sampling configuration. - `source: EvalJSONLFileContentSource { content, type } or EvalJSONLFileIDSource { id, type } or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }` Determines what populates the `item` namespace in this run's data source. - `EvalJSONLFileContentSource object { content, type }` - `content: array of object { item, sample }` The content of the jsonl file. - `item: map[unknown]` - `sample: optional map[unknown]` - `type: "file_content"` The type of jsonl source. Always `file_content`. - `"file_content"` - `EvalJSONLFileIDSource object { id, type }` - `id: string` The identifier of the file. - `type: "file_id"` The type of jsonl source. Always `file_id`. - `"file_id"` - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }` A StoredCompletionsRunDataSource configuration describing a set of filters - `type: "stored_completions"` The type of source. Always `stored_completions`. - `"stored_completions"` - `created_after: optional number or null` An optional Unix timestamp to filter items created after this time. - `created_before: optional number or null` An optional Unix timestamp to filter items created before this time. - `limit: optional number or null` An optional maximum number of items to return. - `metadata: optional Metadata or null` Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `model: optional string or null` An optional model to filter by (e.g., 'gpt-6-astra'). - `type: "completions"` The type of run data source. Always `completions`. - `"completions"` - `input_messages: optional TemplateInputMessages { template, type } or ItemReferenceInputMessages { item_reference, type }` Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - `TemplateInputMessages object { template, type }` - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }` A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - `EasyInputMessage object { content, role, phase, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take precedence over instructions given with the `user` role. Messages with the `assistant` role are presumed to have been generated by the model in previous interactions. - `content: string or ResponseInputMessageContentList` Text, image, or audio input to the model, used to generate a response. Can also contain previous assistant responses. - `TextInput = string` A text input to the model. - `ResponseInputMessageContentList = array of ResponseInputContent` A list of one or many input items to the model, containing different content types. - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. - `text: string` The text input to the model. - `type: "input_text"` The type of the input item. Always `input_text`. - `"input_text"` - `prompt_cache_breakpoint: optional object { mode }` Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. - `mode: "explicit"` The breakpoint mode. Always `explicit`. - `"explicit"` - `ResponseInputImage object { detail, type, file_id, 2 more }` An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision). - `detail: ImageDetail` The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`. - `"low"` - `"high"` - `"auto"` - `"original"` - `type: "input_image"` The type of the input item. Always `input_image`. - `"input_image"` - `file_id: optional string or null` The ID of the file to be sent to the model. - `image_url: optional string or null` The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL. - `prompt_cache_breakpoint: optional object { mode }` Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. - `mode: "explicit"` The breakpoint mode. Always `explicit`. - `"explicit"` - `ResponseInputFile object { type, detail, file_data, 4 more }` A file input to the model. - `type: "input_file"` The type of the input item. Always `input_file`. - `"input_file"` - `detail: optional "auto" or "low" or "high"` The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`. - `"auto"` - `"low"` - `"high"` - `file_data: optional string` The content of the file to be sent to the model. - `file_id: optional string or null` The ID of the file to be sent to the model. - `file_url: optional string` The URL of the file to be sent to the model. - `filename: optional string` The name of the file to be sent to the model. - `prompt_cache_breakpoint: optional object { mode }` Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block. - `mode: "explicit"` The breakpoint mode. Always `explicit`. - `"explicit"` - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or `developer`. - `"user"` - `"assistant"` - `"system"` - `"developer"` - `phase: optional "commentary" or "final_answer" or null` Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`). For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend phase on all assistant messages — dropping it can degrade performance. Not used for user messages. - `"commentary"` - `"final_answer"` - `type: optional "message"` The type of the message input. Always `message`. - `"message"` - `EvalMessageObject object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take precedence over instructions given with the `user` role. Messages with the `assistant` role are presumed to have been generated by the model in previous interactions. - `content: string or ResponseInputText or OutputText { text, type } or 3 more` Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - `TextInput = string` A text input to the model. - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. - `OutputText object { text, type }` A text output from the model. - `text: string` The text output from the model. - `type: "output_text"` The type of the output text. Always `output_text`. - `"output_text"` - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. - `image_url: string` The URL of the image input. - `type: "input_image"` The type of the image input. Always `input_image`. - `"input_image"` - `detail: optional string` The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - `input_audio: object { data, format }` - `data: string` Base64-encoded audio data. - `format: "mp3" or "wav"` The format of the audio data. Currently supported formats are `mp3` and `wav`. - `"mp3"` - `"wav"` - `type: "input_audio"` The type of the input item. Always `input_audio`. - `"input_audio"` - `GraderInputs = array of string or ResponseInputText or OutputText { text, type } or 2 more` A list of inputs, each of which may be either an input text, output text, input image, or input audio object. - `TextInput = string` A text input to the model. - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. - `OutputText object { text, type }` A text output from the model. - `text: string` The text output from the model. - `type: "output_text"` The type of the output text. Always `output_text`. - `"output_text"` - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. - `image_url: string` The URL of the image input. - `type: "input_image"` The type of the image input. Always `input_image`. - `"input_image"` - `detail: optional string` The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or `developer`. - `"user"` - `"assistant"` - `"system"` - `"developer"` - `type: optional "message"` The type of the message input. Always `message`. - `"message"` - `type: "template"` The type of input messages. Always `template`. - `"template"` - `ItemReferenceInputMessages object { item_reference, type }` - `item_reference: string` A reference to a variable in the `item` namespace. Ie, "item.input_trajectory" - `type: "item_reference"` The type of input messages. Always `item_reference`. - `"item_reference"` - `model: optional string` The name of the model to use for generating completions (e.g. "o3-mini"). - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }` - `max_completion_tokens: optional number` The maximum number of tokens in the generated output. - `reasoning_effort: optional ReasoningEffort or null` Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](/api/docs/guides/reasoning) for model-specific support. - `"none"` - `"minimal"` - `"low"` - `"medium"` - `"high"` - `"xhigh"` - `"max"` - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject` An object specifying the format that the model must output. Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](/api/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - `ResponseFormatText object { type }` Default response format. Used to generate text responses. - `type: "text"` The type of response format being defined. Always `text`. - `"text"` - `ResponseFormatJSONSchema object { json_schema, type }` JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](/api/docs/guides/structured-outputs). - `json_schema: object { name, description, schema, strict }` Structured Outputs configuration options, including a JSON Schema. - `name: string` The name of the response format. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64. - `description: optional string` A description of what the response format is for, used by the model to determine how to respond in the format. - `schema: optional map[unknown]` The schema for the response format, described as a JSON Schema object. Learn how to build JSON schemas [here](https://json-schema.org/). - `strict: optional boolean or null` Whether to enable strict schema adherence when generating the output. If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs guide](/api/docs/guides/structured-outputs). - `type: "json_schema"` The type of response format being defined. Always `json_schema`. - `"json_schema"` - `ResponseFormatJSONObject object { type }` JSON object response format. An older method of generating JSON responses. Using `json_schema` is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so. - `type: "json_object"` The type of response format being defined. Always `json_object`. - `"json_object"` - `seed: optional number` A seed value to initialize the randomness, during sampling. - `temperature: optional number` A higher temperature increases randomness in the outputs. - `tools: optional array of ChatCompletionFunctionTool` A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported. - `function: FunctionDefinition` - `name: string` The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64. - `description: optional string` A description of what the function does, used by the model to choose when and how to call the function. - `parameters: optional FunctionParameters` The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict: optional boolean or null` Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling). - `type: "function"` The type of the tool. Currently, only `function` is supported. - `"function"` - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `ResponsesRunDataSource object { source, type, input_messages, 2 more }` A ResponsesRunDataSource object describing a model sampling configuration. - `source: EvalJSONLFileContentSource { content, type } or EvalJSONLFileIDSource { id, type } or EvalResponsesSource { type, created_after, created_before, 8 more }` Determines what populates the `item` namespace in this run's data source. - `EvalJSONLFileContentSource object { content, type }` - `content: array of object { item, sample }` The content of the jsonl file. - `item: map[unknown]` - `sample: optional map[unknown]` - `type: "file_content"` The type of jsonl source. Always `file_content`. - `"file_content"` - `EvalJSONLFileIDSource object { id, type }` - `id: string` The identifier of the file. - `type: "file_id"` The type of jsonl source. Always `file_id`. - `"file_id"` - `EvalResponsesSource object { type, created_after, created_before, 8 more }` A EvalResponsesSource object describing a run data source configuration. - `type: "responses"` The type of run data source. Always `responses`. - `"responses"` - `created_after: optional number or null` Only include items created after this timestamp (inclusive). This is a query parameter used to select responses. - `created_before: optional number or null` Only include items created before this timestamp (inclusive). This is a query parameter used to select responses. - `instructions_search: optional string or null` Optional string to search the 'instructions' field. This is a query parameter used to select responses. - `metadata: optional unknown or null` Metadata filter for the responses. This is a query parameter used to select responses. - `model: optional string or null` The name of the model to find responses for. This is a query parameter used to select responses. - `reasoning_effort: optional ReasoningEffort or null` Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](/api/docs/guides/reasoning) for model-specific support. - `temperature: optional number or null` Sampling temperature. This is a query parameter used to select responses. - `tools: optional array of string or null` List of tool names. This is a query parameter used to select responses. - `top_p: optional number or null` Nucleus sampling parameter. This is a query parameter used to select responses. - `users: optional array of string or null` List of user identifiers. This is a query parameter used to select responses. - `type: "responses"` The type of run data source. Always `responses`. - `"responses"` - `input_messages: optional InputMessagesTemplate { template, type } or InputMessagesItemReference { item_reference, type }` Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - `InputMessagesTemplate object { template, type }` - `template: array of ChatMessage { content, role } or EvalMessageObject { content, role, type }` A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - `ChatMessage object { content, role }` - `content: string` The content of the message. - `role: string` The role of the message (e.g. "system", "assistant", "user"). - `EvalMessageObject object { content, role, type }` A message input to the model with a role indicating instruction following hierarchy. Instructions given with the `developer` or `system` role take precedence over instructions given with the `user` role. Messages with the `assistant` role are presumed to have been generated by the model in previous interactions. - `content: string or ResponseInputText or OutputText { text, type } or 3 more` Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - `TextInput = string` A text input to the model. - `ResponseInputText object { text, type, prompt_cache_breakpoint }` A text input to the model. - `OutputText object { text, type }` A text output from the model. - `text: string` The text output from the model. - `type: "output_text"` The type of the output text. Always `output_text`. - `"output_text"` - `InputImage object { image_url, type, detail }` An image input block used within EvalItem content arrays. - `image_url: string` The URL of the image input. - `type: "input_image"` The type of the image input. Always `input_image`. - `"input_image"` - `detail: optional string` The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`. - `ResponseInputAudio object { input_audio, type }` An audio input to the model. - `GraderInputs = array of string or ResponseInputText or OutputText { text, type } or 2 more` A list of inputs, each of which may be either an input text, output text, input image, or input audio object. - `role: "user" or "assistant" or "system" or "developer"` The role of the message input. One of `user`, `assistant`, `system`, or `developer`. - `"user"` - `"assistant"` - `"system"` - `"developer"` - `type: optional "message"` The type of the message input. Always `message`. - `"message"` - `type: "template"` The type of input messages. Always `template`. - `"template"` - `InputMessagesItemReference object { item_reference, type }` - `item_reference: string` A reference to a variable in the `item` namespace. Ie, "item.name" - `type: "item_reference"` The type of input messages. Always `item_reference`. - `"item_reference"` - `model: optional string` The name of the model to use for generating completions (e.g. "o3-mini"). - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }` - `max_completion_tokens: optional number` The maximum number of tokens in the generated output. - `reasoning_effort: optional ReasoningEffort or null` Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](/api/docs/guides/reasoning) for model-specific support. - `seed: optional number` A seed value to initialize the randomness, during sampling. - `temperature: optional number` A higher temperature increases randomness in the outputs. - `text: optional object { format }` Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - [Text inputs and outputs](/api/docs/guides/text) - [Structured Outputs](/api/docs/guides/structured-outputs) - `format: optional ResponseFormatTextConfig` An object specifying the format that the model must output. Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](/api/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. **Not recommended for gpt-4o and newer models:** Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - `ResponseFormatText object { type }` Default response format. Used to generate text responses. - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }` JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](/api/docs/guides/structured-outputs). - `name: string` The name of the response format. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64. - `schema: map[unknown]` The schema for the response format, described as a JSON Schema object. Learn how to build JSON schemas [here](https://json-schema.org/). - `type: "json_schema"` The type of response format being defined. Always `json_schema`. - `"json_schema"` - `description: optional string` A description of what the response format is for, used by the model to determine how to respond in the format. - `strict: optional boolean or null` Whether to enable strict schema adherence when generating the output. If set to true, the model will always follow the exact schema defined in the `schema` field. Only a subset of JSON Schema is supported when `strict` is `true`. To learn more, read the [Structured Outputs guide](/api/docs/guides/structured-outputs). - `ResponseFormatJSONObject object { type }` JSON object response format. An older method of generating JSON responses. Using `json_schema` is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so. - `tools: optional array of Function { name, parameters, strict, 6 more } or FileSearch { type, vector_store_ids, filters, 2 more } or Computer { type } or 13 more` An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the model's capabilities, like [web search](/api/docs/guides/tools-web-search) or [file search](/api/docs/guides/tools-file-search). Learn more about [built-in tools](/api/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about [function calling](/api/docs/guides/function-calling). - `Function object { name, parameters, strict, 6 more }` Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling). - `name: string` The name of the function to call. - `parameters: map[unknown] or null` A JSON schema object describing the parameters of the function. - `strict: boolean or null` Whether strict parameter validation is enforced for this function tool. - `type: "function"` The type of the function tool. Always `function`. - `"function"` - `allowed_callers: optional array of "direct" or "programmatic" or null` The tool invocation context(s). - `"direct"` - `"programmatic"` - `async: optional boolean` - `defer_loading: optional boolean` Whether this function is deferred and loaded via tool search. - `description: optional string or null` A description of the function. Used by the model to determine whether or not to call the function. - `output_schema: optional map[unknown] or null` A JSON schema object describing the JSON value encoded in string outputs for this function. - `FileSearch object { type, vector_store_ids, filters, 2 more }` A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search). - `type: "file_search"` The type of the file search tool. Always `file_search`. - `"file_search"` - `vector_store_ids: array of string` The IDs of the vector stores to search. - `filters: optional ComparisonFilter or CompoundFilter or null` A filter to apply. - `ComparisonFilter object { key, type, value }` A filter used to compare a specified attribute key to a given value using a defined comparison operation. - `key: string` The key to compare against the value. - `type: "eq" or "ne" or "gt" or 5 more` Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`. - `eq`: equals - `ne`: not equal - `gt`: greater than - `gte`: greater than or equal - `lt`: less than - `lte`: less than or equal - `in`: in - `nin`: not in - `"eq"` - `"ne"` - `"gt"` - `"gte"` - `"lt"` - `"lte"` - `"in"` - `"nin"` - `value: string or number or boolean or array of string or number` The value to compare against the attribute key; supports string, number, or boolean types. - `string` - `number` - `boolean` - `array of string or number` - `string` - `number` - `CompoundFilter object { filters, type }` Combine multiple filters using `and` or `or`. - `filters: array of ComparisonFilter or CompoundFilter` Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`. - `ComparisonFilter object { key, type, value }` A filter used to compare a specified attribute key to a given value using a defined comparison operation. - `CompoundFilter object { filters, type }` Combine multiple filters using `and` or `or`. - `type: "and" or "or"` Type of operation: `and` or `or`. - `"and"` - `"or"` - `max_num_results: optional number` The maximum number of results to return. This number should be between 1 and 50 inclusive. - `ranking_options: optional object { hybrid_search, ranker, score_threshold }` Ranking options for search. - `hybrid_search: optional object { embedding_weight, text_weight }` Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled. - `embedding_weight: number` The weight of the embedding in the reciprocal ranking fusion. - `text_weight: number` The weight of the text in the reciprocal ranking fusion. - `ranker: optional "auto" or "default-2024-11-15"` The ranker to use for the file search. - `"auto"` - `"default-2024-11-15"` - `score_threshold: optional number` The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results. - `Computer object { type }` A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use). - `type: "computer"` The type of the computer tool. Always `computer`. - `"computer"` - `ComputerUsePreview object { display_height, display_width, environment, type }` A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use). - `display_height: number` The height of the computer display. - `display_width: number` The width of the computer display. - `environment: "windows" or "mac" or "linux" or 2 more` The type of computer environment to control. - `"windows"` - `"mac"` - `"linux"` - `"ubuntu"` - `"browser"` - `type: "computer_use_preview"` The type of the computer use tool. Always `computer_use_preview`. - `"computer_use_preview"` - `WebSearch object { type, external_web_access, filters, 2 more }` Search the Internet for sources related to the prompt. Learn more about the [web search tool](/api/docs/guides/tools-web-search). - `type: "web_search" or "web_search_2025_08_26"` The type of the web search tool. One of `web_search` or `web_search_2025_08_26`. - `"web_search"` - `"web_search_2025_08_26"` - `external_web_access: optional boolean` Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content. - `filters: optional object { allowed_domains } or null` Filters for the search. - `allowed_domains: optional array of string or null` Allowed domains for the search. If not provided, all domains are allowed. Subdomains of the provided domains are allowed as well. Example: `["pubmed.ncbi.nlm.nih.gov"]` - `search_context_size: optional "low" or "medium" or "high"` High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default. - `"low"` - `"medium"` - `"high"` - `user_location: optional object { city, country, region, 2 more } or null` The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields. - `city: optional string or null` Free text input for the city of the user, e.g. `San Francisco`. - `country: optional string or null` The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`. - `region: optional string or null` Free text input for the region of the user, e.g. `California`. - `timezone: optional string or null` The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - `type: optional "approximate"` The type of location approximation. Always `approximate`. - `"approximate"` - `Mcp object { server_label, type, allowed_callers, 9 more }` Give the model access to additional tools via remote Model Context Protocol (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp). - `server_label: string` A label for this MCP server, used to identify it in tool calls. - `type: "mcp"` The type of the MCP tool. Always `mcp`. - `"mcp"` - `allowed_callers: optional array of "direct" or "programmatic" or null` The tool invocation context(s). - `"direct"` - `"programmatic"` - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names } or null` List of allowed tool names or a filter object. - `McpAllowedTools = array of string` A string array of allowed tool names - `McpToolFilter object { read_only, tool_names }` A filter object to specify which tools are allowed. - `read_only: optional boolean` Indicates whether or not a tool modifies data or is read-only. If an MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint), it will match this filter. - `tool_names: optional array of string` List of allowed tool names. - `authorization: optional string` An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here. - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more` Identifier for service connectors, like those available in ChatGPT. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors). This field is deprecated for models released after September 1, 2026. Use `server_url` to connect to a remote MCP server, or `tunnel_id` to connect through a Secure MCP Tunnel. Currently supported `connector_id` values are: - Dropbox: `connector_dropbox` - Gmail: `connector_gmail` - Google Calendar: `connector_googlecalendar` - Google Drive: `connector_googledrive` - Microsoft Teams: `connector_microsoftteams` - Outlook Calendar: `connector_outlookcalendar` - Outlook Email: `connector_outlookemail` - SharePoint: `connector_sharepoint` - `"connector_dropbox"` - `"connector_gmail"` - `"connector_googlecalendar"` - `"connector_googledrive"` - `"connector_microsoftteams"` - `"connector_outlookcalendar"` - `"connector_outlookemail"` - `"connector_sharepoint"` - `defer_loading: optional boolean` Whether this MCP tool is deferred and discovered via tool search. - `headers: optional map[string] or null` Optional HTTP headers to send to the MCP server. Use for authentication or other purposes. - `require_approval: optional McpToolApprovalFilter { always, never } or "always" or "never" or null` Specify which of the MCP server's tools require approval. - `McpToolApprovalFilter object { always, never }` Specify which of the MCP server's tools require approval. Can be `always`, `never`, or a filter object associated with tools that require approval. - `always: optional object { read_only, tool_names }` A filter object to specify which tools are allowed. - `read_only: optional boolean` Indicates whether or not a tool modifies data or is read-only. If an MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint), it will match this filter. - `tool_names: optional array of string` List of allowed tool names. - `never: optional object { read_only, tool_names }` A filter object to specify which tools are allowed. - `read_only: optional boolean` Indicates whether or not a tool modifies data or is read-only. If an MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint), it will match this filter. - `tool_names: optional array of string` List of allowed tool names. - `McpToolApprovalSetting = "always" or "never"` Specify a single approval policy for all tools. One of `always` or `never`. When set to `always`, all tools will require approval. When set to `never`, all tools will not require approval. - `"always"` - `"never"` - `server_description: optional string` Optional description of the MCP server, used to provide more context. - `server_url: optional string` The URL for the MCP server. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - `tunnel_id: optional string` The Secure MCP Tunnel ID to use instead of a direct server URL. One of `server_url`, `connector_id`, or `tunnel_id` must be provided. - `CodeInterpreter object { container, type, allowed_callers }` A tool that runs Python code to help generate a response to a prompt. - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }` The code interpreter container. Can be a container ID or an object that specifies uploaded file IDs to make available to your code, along with an optional `memory_limit` setting. - `string` The container ID. - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }` Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on. - `type: "auto"` Always `auto`. - `"auto"` - `file_ids: optional array of string` An optional list of uploaded files to make available to your code. - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null` The memory limit for the code interpreter container. - `"1g"` - `"4g"` - `"16g"` - `"64g"` - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist` Network access policy for the container. - `ContainerNetworkPolicyDisabled object { type }` - `type: "disabled"` Disable outbound network access. Always `disabled`. - `"disabled"` - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }` - `allowed_domains: array of string` A list of allowed domains when type is `allowlist`. - `type: "allowlist"` Allow outbound network access only to specified domains. Always `allowlist`. - `"allowlist"` - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret` Optional domain-scoped secrets for allowlisted domains. - `domain: string` The domain associated with the secret. - `name: string` The name of the secret to inject for the domain. - `value: string` The secret value to inject for the domain. - `type: "code_interpreter"` The type of the code interpreter tool. Always `code_interpreter`. - `"code_interpreter"` - `allowed_callers: optional array of "direct" or "programmatic" or null` The tool invocation context(s). - `"direct"` - `"programmatic"` - `ProgrammaticToolCalling object { type }` - `type: "programmatic_tool_calling"` The type of the tool. Always `programmatic_tool_calling`. - `"programmatic_tool_calling"` - `ImageGeneration object { type, action, background, 9 more }` A tool that generates images using the GPT image models. - `type: "image_generation"` The type of the image generation tool. Always `image_generation`. - `"image_generation"` - `action: optional "generate" or "edit" or "auto"` Whether to generate a new image or edit an existing image. Default: `auto`. - `"generate"` - `"edit"` - `"auto"` - `background: optional "transparent" or "opaque" or "auto"` Set the background of the generated image. One of `transparent`, `opaque`, or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including their `2026-09-08` snapshots, support `opaque` and `transparent` backgrounds. Transparent backgrounds are available for supported GPT Image models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in preview. When using `transparent`, set the output format to `png` or `webp`. Default: `auto`. - `"transparent"` - `"opaque"` - `"auto"` - `input_fidelity: optional "high" or "low" or null` Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models. - `"high"` - `"low"` - `input_image_mask: optional object { file_id, image_url }` Optional mask for inpainting. Contains `image_url` (string, optional) and `file_id` (string, optional). - `file_id: optional string` File ID for the mask image. - `image_url: optional string` Base64-encoded mask image. - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more` The image generation model to use. One of `gpt-image-1`, `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default: `gpt-image-1`. - `string` - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more` The image generation model to use. One of `gpt-image-1`, `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default: `gpt-image-1`. - `"gpt-image-1"` - `"gpt-image-1-mini"` - `"gpt-image-1.5"` - `"gpt-image-2"` - `"gpt-image-2-2026-04-21"` - `"gpt-image-2.5-sunburst"` - `"gpt-image-2.5-sunburst-2026-09-08"` - `"gpt-image-2.5-flare"` - `"gpt-image-2.5-flare-2026-09-08"` - `moderation: optional "auto" or "low"` Moderation level for the generated image. Default: `auto`. - `"auto"` - `"low"` - `output_compression: optional number` Compression level for the output image. Default: 100. - `output_format: optional "png" or "webp" or "jpeg"` The output format of the generated image. One of `png`, `webp`, or `jpeg`. Default: `png`. - `"png"` - `"webp"` - `"jpeg"` - `partial_images: optional number` Number of partial images to generate in streaming mode, from 0 (default value) to 3. - `quality: optional "low" or "medium" or "high" or 3 more` The quality of the generated image. The GPT image models support `low`, `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including their `2026-09-08` snapshots, also support `xhigh` and `max`. Default: `auto`. - `"low"` - `"medium"` - `"high"` - `"xhigh"` - `"max"` - `"auto"` - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"` The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing. - `string` - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"` The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing. - `"1024x1024"` - `"1024x1536"` - `"1536x1024"` - `"auto"` - `LocalShell object { type }` A tool that allows the model to execute shell commands in a local environment. - `type: "local_shell"` The type of the local shell tool. Always `local_shell`. - `"local_shell"` - `Shell object { type, allowed_callers, environment }` A tool that allows the model to execute shell commands. - `type: "shell"` The type of the shell tool. Always `shell`. - `"shell"` - `allowed_callers: optional array of "direct" or "programmatic" or null` The tool invocation context(s). - `"direct"` - `"programmatic"` - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null` - `ContainerAuto object { type, file_ids, memory_limit, 2 more }` - `type: "container_auto"` Automatically creates a container for this request - `"container_auto"` - `file_ids: optional array of string` An optional list of uploaded files to make available to your code. - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null` The memory limit for the container. - `"1g"` - `"4g"` - `"16g"` - `"64g"` - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist` Network access policy for the container. - `ContainerNetworkPolicyDisabled object { type }` - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }` - `skills: optional array of SkillReference or InlineSkill` An optional list of skills referenced by id or inline data. - `SkillReference object { skill_id, type, version }` - `skill_id: string` The ID of the referenced skill. - `type: "skill_reference"` References a skill created with the /v1/skills endpoint. - `"skill_reference"` - `version: optional string` Optional skill version. Use a positive integer or 'latest'. Omit for default. - `InlineSkill object { description, name, source, type }` - `description: string` The description of the skill. - `name: string` The name of the skill. - `source: InlineSkillSource` Inline skill payload - `data: string` Base64-encoded skill zip bundle. - `media_type: "application/zip"` The media type of the inline skill payload. Must be `application/zip`. - `"application/zip"` - `type: "base64"` The type of the inline skill source. Must be `base64`. - `"base64"` - `type: "inline"` Defines an inline skill for this request. - `"inline"` - `LocalEnvironment object { type, skills }` - `type: "local"` Use a local computer environment. - `"local"` - `skills: optional array of LocalSkill` An optional list of skills. - `description: string` The description of the skill. - `name: string` The name of the skill. - `path: string` The path to the directory containing the skill. - `ContainerReference object { container_id, type }` - `container_id: string` The ID of the referenced container. - `type: "container_reference"` References a container created with the /v1/containers endpoint - `"container_reference"` - `Custom object { name, type, allowed_callers, 4 more }` A custom tool that processes input using a specified format. Learn more about [custom tools](/api/docs/guides/function-calling#custom-tools) - `name: string` The name of the custom tool, used to identify it in tool calls. - `type: "custom"` The type of the custom tool. Always `custom`. - `"custom"` - `allowed_callers: optional array of "direct" or "programmatic" or null` The tool invocation context(s). - `"direct"` - `"programmatic"` - `async: optional boolean` Whether the tool response can be returned asynchronously versus immediately returned on next response creation. - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. - `description: optional string` Optional description of the custom tool, used to provide more context. - `format: optional CustomToolInputFormat` The input format for the custom tool. Default is unconstrained text. - `Text object { type }` Unconstrained free-form text. - `type: "text"` Unconstrained text format. Always `text`. - `"text"` - `Grammar object { definition, syntax, type }` A grammar defined by the user. - `definition: string` The grammar definition. - `syntax: "lark" or "regex"` The syntax of the grammar definition. One of `lark` or `regex`. - `"lark"` - `"regex"` - `type: "grammar"` Grammar format. Always `grammar`. - `"grammar"` - `Namespace object { description, name, tools, type }` Groups function/custom tools under a shared namespace. - `description: string` A description of the namespace shown to the model. - `name: string` The namespace name used in tool calls (for example, `crm`). - `tools: array of Function { name, type, allowed_callers, 6 more } or Custom { name, type, allowed_callers, 4 more }` The function/custom tools available inside this namespace. - `Function object { name, type, allowed_callers, 6 more }` - `name: string` - `type: "function"` - `"function"` - `allowed_callers: optional array of "direct" or "programmatic" or null` The tool invocation context(s). - `"direct"` - `"programmatic"` - `async: optional boolean` Whether the tool response can be returned asynchronously versus immediately returned on next response creation. - `defer_loading: optional boolean` Whether this function should be deferred and discovered via tool search. - `description: optional string or null` - `output_schema: optional map[unknown] or null` A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs. - `parameters: optional unknown or null` - `strict: optional boolean or null` Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise. - `Custom object { name, type, allowed_callers, 4 more }` A custom tool that processes input using a specified format. Learn more about [custom tools](/api/docs/guides/function-calling#custom-tools) - `name: string` The name of the custom tool, used to identify it in tool calls. - `type: "custom"` The type of the custom tool. Always `custom`. - `"custom"` - `allowed_callers: optional array of "direct" or "programmatic" or null` The tool invocation context(s). - `"direct"` - `"programmatic"` - `async: optional boolean` Whether the tool response can be returned asynchronously versus immediately returned on next response creation. - `defer_loading: optional boolean` Whether this tool should be deferred and discovered via tool search. - `description: optional string` Optional description of the custom tool, used to provide more context. - `format: optional CustomToolInputFormat` The input format for the custom tool. Default is unconstrained text. - `type: "namespace"` The type of the tool. Always `namespace`. - `"namespace"` - `ToolSearch object { type, description, execution, parameters }` Hosted or BYOT tool search configuration for deferred tools. - `type: "tool_search"` The type of the tool. Always `tool_search`. - `"tool_search"` - `description: optional string or null` Description shown to the model for a client-executed tool search tool. - `execution: optional "server" or "client"` Whether tool search is executed by the server or by the client. - `"server"` - `"client"` - `parameters: optional unknown or null` Parameter schema for a client-executed tool search tool. - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }` This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search). - `type: "web_search_preview" or "web_search_preview_2025_03_11"` The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`. - `"web_search_preview"` - `"web_search_preview_2025_03_11"` - `search_content_types: optional array of "text" or "image"` - `"text"` - `"image"` - `search_context_size: optional "low" or "medium" or "high"` High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default. - `"low"` - `"medium"` - `"high"` - `user_location: optional object { type, city, country, 2 more } or null` The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields. - `type: "approximate"` The type of location approximation. Always `approximate`. - `"approximate"` - `city: optional string or null` Free text input for the city of the user, e.g. `San Francisco`. - `country: optional string or null` The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`. - `region: optional string or null` Free text input for the region of the user, e.g. `California`. - `timezone: optional string or null` The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`. - `ApplyPatch object { type, allowed_callers }` Allows the assistant to create, delete, or update files using unified diffs. - `type: "apply_patch"` The type of the tool. Always `apply_patch`. - `"apply_patch"` - `allowed_callers: optional array of "direct" or "programmatic" or null` The tool invocation context(s). - `"direct"` - `"programmatic"` - `top_p: optional number` An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `error: EvalAPIError or null` An object representing an error response from the Eval API. - `code: string` The error code. - `message: string` The error message. - `eval_id: string` The identifier of the associated evaluation. - `metadata: Metadata or null` Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `model: string or null` The model that is evaluated, if applicable. - `name: string or null` The name of the evaluation run. - `object: "eval.run"` The type of the object. Always "eval.run". - `"eval.run"` - `per_model_usage: array of object { cached_tokens, completion_tokens, invocation_count, 3 more } or null` Usage statistics for each model during the evaluation run. - `cached_tokens: number` The number of tokens retrieved from cache. - `completion_tokens: number` The number of completion tokens generated. - `invocation_count: number` The number of invocations. - `model_name: string` The name of the model. - `prompt_tokens: number` The number of prompt tokens used. - `total_tokens: number` The total number of tokens used. - `per_testing_criteria_results: array of object { failed, passed, testing_criteria } or null` Results per testing criteria applied during the evaluation run. - `failed: number` Number of tests failed for this criteria. - `passed: number` Number of tests passed for this criteria. - `testing_criteria: string` A description of the testing criteria. - `report_url: string` The URL to the rendered evaluation run report on the UI dashboard. - `result_counts: object { errored, failed, passed, total }` Counters summarizing the outcomes of the evaluation run. - `errored: number` Number of output items that resulted in an error. - `failed: number` Number of output items that failed to pass the evaluation. - `passed: number` Number of output items that passed the evaluation. - `total: number` Total number of executed output items. - `status: string` The status of the evaluation run. - `first_id: string` The identifier of the first eval run in the data array. - `has_more: boolean` Indicates whether there are more evals available. - `last_id: string` The identifier of the last eval run in the data array. - `object: "list"` The type of this object. It is always set to "list". - `"list"` ### Example ```http curl https://api.openai.com/v1/evals/$EVAL_ID/runs \ -H "Authorization: Bearer $OPENAI_API_KEY" ``` #### Response ```json { "data": [ { "id": "id", "created_at": 0, "data_source": { "source": { "content": [ { "item": { "foo": "bar" }, "sample": { "foo": "bar" } } ], "type": "file_content" }, "type": "jsonl" }, "error": { "code": "code", "message": "message" }, "eval_id": "eval_id", "metadata": { "foo": "string" }, "model": "model", "name": "name", "object": "eval.run", "per_model_usage": [ { "cached_tokens": 0, "completion_tokens": 0, "invocation_count": 0, "model_name": "model_name", "prompt_tokens": 0, "total_tokens": 0 } ], "per_testing_criteria_results": [ { "failed": 0, "passed": 0, "testing_criteria": "testing_criteria" } ], "report_url": "https://example.com", "result_counts": { "errored": 0, "failed": 0, "passed": 0, "total": 0 }, "status": "status" } ], "first_id": "first_id", "has_more": true, "last_id": "last_id", "object": "list" } ``` ### Example ```http curl https://api.openai.com/v1/evals/egroup_67abd54d9b0081909a86353f6fb9317a/runs \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" ``` #### Response ```json { "object": "list", "data": [ { "object": "eval.run", "id": "evalrun_67e0c7d31560819090d60c0780591042", "eval_id": "eval_67e0c726d560819083f19a957c4c640b", "report_url": "https://platform.openai.com/evaluations/eval_67e0c726d560819083f19a957c4c640b", "status": "completed", "model": "o3-mini", "name": "bulk_with_negative_examples_o3-mini", "created_at": 1742784467, "result_counts": { "total": 1, "errored": 0, "failed": 0, "passed": 1 }, "per_model_usage": [ { "model_name": "o3-mini", "invocation_count": 1, "prompt_tokens": 563, "completion_tokens": 874, "total_tokens": 1437, "cached_tokens": 0 } ], "per_testing_criteria_results": [ { "testing_criteria": "Push Notification Summary Grader-1808cd0b-eeec-4e0b-a519-337e79f4f5d1", "passed": 1, "failed": 0 } ], "data_source": { "type": "completions", "source": { "type": "file_content", "content": [ { "item": { "notifications": "\n- New message from Sarah: \"Can you call me later?\"\n- Your package has been delivered!\n- Flash sale: 20% off electronics for the next 2 hours!\n" } } ] }, "input_messages": { "type": "template", "template": [ { "type": "message", "role": "developer", "content": { "type": "input_text", "text": "\n\n\n\nYou are a helpful assistant that takes in an array of push notifications and returns a collapsed summary of them.\nThe push notification will be provided as follows:\n\n...notificationlist...\n\n\nYou should return just the summary and nothing else.\n\n\nYou should return a summary that is concise and snappy.\n\n\nHere is an example of a good summary:\n\n- Traffic alert: Accident reported on Main Street.- Package out for delivery: Expected by 5 PM.- New friend suggestion: Connect with Emma.\n\n\nTraffic alert, package expected by 5pm, suggestion for new friend (Emily).\n\n\n\nHere is an example of a bad summary:\n\n- Traffic alert: Accident reported on Main Street.- Package out for delivery: Expected by 5 PM.- New friend suggestion: Connect with Emma.\n\n\nTraffic alert reported on main street. You have a package that will arrive by 5pm, Emily is a new friend suggested for you.\n\n" } }, { "type": "message", "role": "user", "content": { "type": "input_text", "text": "{{item.notifications}}" } } ] }, "model": "o3-mini" }, "error": null, "metadata": {} } ], "first_id": "evalrun_67e0c7d31560819090d60c0780591042", "last_id": "evalrun_67e0c7d31560819090d60c0780591042", "has_more": true } ```