# Runs

## Cancel eval run

**post** `/evals/{eval_id}/runs/{run_id}/cancel`

Cancel an ongoing evaluation run.

### Path Parameters

- `eval_id: string`

- `run_id: string`

### Returns

- `id: string`

  Unique identifier for the evaluation run.

- `created_at: number`

  Unix timestamp (in seconds) when the evaluation run was created.

- `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }`

  Information about the run's data source.

  - `CreateEvalJSONLRunDataSource object { source, type }`

    A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

      Determines what populates the `item` namespace in the data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

    - `type: "jsonl"`

      The type of data source. Always `jsonl`.

      - `"jsonl"`

  - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

    A CompletionsRunDataSource object describing a model sampling configuration.

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

      Determines what populates the `item` namespace in this run's data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

      - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

        A StoredCompletionsRunDataSource configuration describing a set of filters

        - `type: "stored_completions"`

          The type of source. Always `stored_completions`.

          - `"stored_completions"`

        - `created_after: optional number or null`

          An optional Unix timestamp to filter items created after this time.

        - `created_before: optional number or null`

          An optional Unix timestamp to filter items created before this time.

        - `limit: optional number or null`

          An optional maximum number of items to return.

        - `metadata: optional Metadata or null`

          Set of 16 key-value pairs that can be attached to an object. This can be
          useful for storing additional information about the object in a structured
          format, and querying for objects via API or the dashboard.

          Keys are strings with a maximum length of 64 characters. Values are strings
          with a maximum length of 512 characters.

        - `model: optional string or null`

          An optional model to filter by (e.g., 'gpt-6-astra').

    - `type: "completions"`

      The type of run data source. Always `completions`.

      - `"completions"`

    - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

      Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

      - `TemplateInputMessages object { template, type }`

        - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

          A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

          - `EasyInputMessage object { content, role, phase, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputMessageContentList`

              Text, image, or audio input to the model, used to generate a response.
              Can also contain previous assistant responses.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputMessageContentList = array of ResponseInputContent`

                A list of one or many input items to the model, containing different content
                types.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                  - `text: string`

                    The text input to the model.

                  - `type: "input_text"`

                    The type of the input item. Always `input_text`.

                    - `"input_text"`

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

                - `ResponseInputImage object { detail, type, file_id, 2 more }`

                  An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                  - `detail: ImageDetail`

                    The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                    - `"low"`

                    - `"high"`

                    - `"auto"`

                    - `"original"`

                  - `type: "input_image"`

                    The type of the input item. Always `input_image`.

                    - `"input_image"`

                  - `file_id: optional string or null`

                    The ID of the file to be sent to the model.

                  - `image_url: optional string or null`

                    The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

                - `ResponseInputFile object { type, detail, file_data, 4 more }`

                  A file input to the model.

                  - `type: "input_file"`

                    The type of the input item. Always `input_file`.

                    - `"input_file"`

                  - `detail: optional "auto" or "low" or "high"`

                    The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                    - `"auto"`

                    - `"low"`

                    - `"high"`

                  - `file_data: optional string`

                    The content of the file to be sent to the model.

                  - `file_id: optional string or null`

                    The ID of the file to be sent to the model.

                  - `file_url: optional string`

                    The URL of the file to be sent to the model.

                  - `filename: optional string`

                    The name of the file to be sent to the model.

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `phase: optional "commentary" or "final_answer" or null`

              Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
              For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
              phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

              - `"commentary"`

              - `"final_answer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

          - `EvalMessageObject object { content, role, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

              Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

              - `OutputText object { text, type }`

                A text output from the model.

                - `text: string`

                  The text output from the model.

                - `type: "output_text"`

                  The type of the output text. Always `output_text`.

                  - `"output_text"`

              - `InputImage object { image_url, type, detail }`

                An image input block used within EvalItem content arrays.

                - `image_url: string`

                  The URL of the image input.

                - `type: "input_image"`

                  The type of the image input. Always `input_image`.

                  - `"input_image"`

                - `detail: optional string`

                  The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

              - `ResponseInputAudio object { input_audio, type }`

                An audio input to the model.

                - `input_audio: object { data, format }`

                  - `data: string`

                    Base64-encoded audio data.

                  - `format: "mp3" or "wav"`

                    The format of the audio data. Currently supported formats are `mp3` and
                    `wav`.

                    - `"mp3"`

                    - `"wav"`

                - `type: "input_audio"`

                  The type of the input item. Always `input_audio`.

                  - `"input_audio"`

              - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                A list of inputs, each of which may be either an input text, output text, input
                image, or input audio object.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

        - `type: "template"`

          The type of input messages. Always `template`.

          - `"template"`

      - `ItemReferenceInputMessages object { item_reference, type }`

        - `item_reference: string`

          A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

        - `type: "item_reference"`

          The type of input messages. Always `item_reference`.

          - `"item_reference"`

    - `model: optional string`

      The name of the model to use for generating completions (e.g. "o3-mini").

    - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

      - `max_completion_tokens: optional number`

        The maximum number of tokens in the generated output.

      - `reasoning_effort: optional ReasoningEffort or null`

        Constrains effort on reasoning for reasoning models. Currently supported
        values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
        Reducing reasoning effort can result in faster responses and fewer tokens
        used on reasoning in a response. Not all reasoning models support every
        value. See the
        [reasoning guide](/api/docs/guides/reasoning)
        for model-specific support.

        - `"none"`

        - `"minimal"`

        - `"low"`

        - `"medium"`

        - `"high"`

        - `"xhigh"`

        - `"max"`

      - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

        An object specifying the format that the model must output.

        Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
        Structured Outputs which ensures the model will match your supplied JSON
        schema. Learn more in the [Structured Outputs
        guide](/api/docs/guides/structured-outputs).

        Setting to `{ "type": "json_object" }` enables the older JSON mode, which
        ensures the message the model generates is valid JSON. Using `json_schema`
        is preferred for models that support it.

        - `ResponseFormatText object { type }`

          Default response format. Used to generate text responses.

          - `type: "text"`

            The type of response format being defined. Always `text`.

            - `"text"`

        - `ResponseFormatJSONSchema object { json_schema, type }`

          JSON Schema response format. Used to generate structured JSON responses.
          Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

          - `json_schema: object { name, description, schema, strict }`

            Structured Outputs configuration options, including a JSON Schema.

            - `name: string`

              The name of the response format. Must be a-z, A-Z, 0-9, or contain
              underscores and dashes, with a maximum length of 64.

            - `description: optional string`

              A description of what the response format is for, used by the model to
              determine how to respond in the format.

            - `schema: optional map[unknown]`

              The schema for the response format, described as a JSON Schema object.
              Learn how to build JSON schemas [here](https://json-schema.org/).

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the output.
              If set to true, the model will always follow the exact schema defined
              in the `schema` field. Only a subset of JSON Schema is supported when
              `strict` is `true`. To learn more, read the [Structured Outputs
              guide](/api/docs/guides/structured-outputs).

          - `type: "json_schema"`

            The type of response format being defined. Always `json_schema`.

            - `"json_schema"`

        - `ResponseFormatJSONObject object { type }`

          JSON object response format. An older method of generating JSON responses.
          Using `json_schema` is recommended for models that support it. Note that the
          model will not generate JSON without a system or user message instructing it
          to do so.

          - `type: "json_object"`

            The type of response format being defined. Always `json_object`.

            - `"json_object"`

      - `seed: optional number`

        A seed value to initialize the randomness, during sampling.

      - `temperature: optional number`

        A higher temperature increases randomness in the outputs.

      - `tools: optional array of ChatCompletionFunctionTool`

        A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

        - `function: FunctionDefinition`

          - `name: string`

            The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

          - `description: optional string`

            A description of what the function does, used by the model to choose when and how to call the function.

          - `parameters: optional FunctionParameters`

            The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

            Omitting `parameters` defines a function with an empty parameter list.

          - `strict: optional boolean or null`

            Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

        - `type: "function"`

          The type of the tool. Currently, only `function` is supported.

          - `"function"`

      - `top_p: optional number`

        An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

  - `ResponsesRunDataSource object { source, type, input_messages, 2 more }`

    A ResponsesRunDataSource object describing a model sampling configuration.

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or EvalResponsesSource { type, created_after, created_before, 8 more }`

      Determines what populates the `item` namespace in this run's data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

      - `EvalResponsesSource object { type, created_after, created_before, 8 more }`

        A EvalResponsesSource object describing a run data source configuration.

        - `type: "responses"`

          The type of run data source. Always `responses`.

          - `"responses"`

        - `created_after: optional number or null`

          Only include items created after this timestamp (inclusive). This is a query parameter used to select responses.

        - `created_before: optional number or null`

          Only include items created before this timestamp (inclusive). This is a query parameter used to select responses.

        - `instructions_search: optional string or null`

          Optional string to search the 'instructions' field. This is a query parameter used to select responses.

        - `metadata: optional unknown or null`

          Metadata filter for the responses. This is a query parameter used to select responses.

        - `model: optional string or null`

          The name of the model to find responses for. This is a query parameter used to select responses.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

        - `temperature: optional number or null`

          Sampling temperature. This is a query parameter used to select responses.

        - `tools: optional array of string or null`

          List of tool names. This is a query parameter used to select responses.

        - `top_p: optional number or null`

          Nucleus sampling parameter. This is a query parameter used to select responses.

        - `users: optional array of string or null`

          List of user identifiers. This is a query parameter used to select responses.

    - `type: "responses"`

      The type of run data source. Always `responses`.

      - `"responses"`

    - `input_messages: optional InputMessagesTemplate { template, type }  or InputMessagesItemReference { item_reference, type }`

      Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

      - `InputMessagesTemplate object { template, type }`

        - `template: array of ChatMessage { content, role }  or EvalMessageObject { content, role, type }`

          A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

          - `ChatMessage object { content, role }`

            - `content: string`

              The content of the message.

            - `role: string`

              The role of the message (e.g. "system", "assistant", "user").

          - `EvalMessageObject object { content, role, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

              Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

              - `OutputText object { text, type }`

                A text output from the model.

                - `text: string`

                  The text output from the model.

                - `type: "output_text"`

                  The type of the output text. Always `output_text`.

                  - `"output_text"`

              - `InputImage object { image_url, type, detail }`

                An image input block used within EvalItem content arrays.

                - `image_url: string`

                  The URL of the image input.

                - `type: "input_image"`

                  The type of the image input. Always `input_image`.

                  - `"input_image"`

                - `detail: optional string`

                  The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

              - `ResponseInputAudio object { input_audio, type }`

                An audio input to the model.

              - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                A list of inputs, each of which may be either an input text, output text, input
                image, or input audio object.

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

        - `type: "template"`

          The type of input messages. Always `template`.

          - `"template"`

      - `InputMessagesItemReference object { item_reference, type }`

        - `item_reference: string`

          A reference to a variable in the `item` namespace. Ie, "item.name"

        - `type: "item_reference"`

          The type of input messages. Always `item_reference`.

          - `"item_reference"`

    - `model: optional string`

      The name of the model to use for generating completions (e.g. "o3-mini").

    - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }`

      - `max_completion_tokens: optional number`

        The maximum number of tokens in the generated output.

      - `reasoning_effort: optional ReasoningEffort or null`

        Constrains effort on reasoning for reasoning models. Currently supported
        values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
        Reducing reasoning effort can result in faster responses and fewer tokens
        used on reasoning in a response. Not all reasoning models support every
        value. See the
        [reasoning guide](/api/docs/guides/reasoning)
        for model-specific support.

      - `seed: optional number`

        A seed value to initialize the randomness, during sampling.

      - `temperature: optional number`

        A higher temperature increases randomness in the outputs.

      - `text: optional object { format }`

        Configuration options for a text response from the model. Can be plain
        text or structured JSON data. Learn more:

        - [Text inputs and outputs](/api/docs/guides/text)
        - [Structured Outputs](/api/docs/guides/structured-outputs)

        - `format: optional ResponseFormatTextConfig`

          An object specifying the format that the model must output.

          Configuring `{ "type": "json_schema" }` enables Structured Outputs,
          which ensures the model will match your supplied JSON schema. Learn more in the
          [Structured Outputs guide](/api/docs/guides/structured-outputs).

          The default format is `{ "type": "text" }` with no additional options.

          **Not recommended for gpt-4o and newer models:**

          Setting to `{ "type": "json_object" }` enables the older JSON mode, which
          ensures the message the model generates is valid JSON. Using `json_schema`
          is preferred for models that support it.

          - `ResponseFormatText object { type }`

            Default response format. Used to generate text responses.

          - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }`

            JSON Schema response format. Used to generate structured JSON responses.
            Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

            - `name: string`

              The name of the response format. Must be a-z, A-Z, 0-9, or contain
              underscores and dashes, with a maximum length of 64.

            - `schema: map[unknown]`

              The schema for the response format, described as a JSON Schema object.
              Learn how to build JSON schemas [here](https://json-schema.org/).

            - `type: "json_schema"`

              The type of response format being defined. Always `json_schema`.

              - `"json_schema"`

            - `description: optional string`

              A description of what the response format is for, used by the model to
              determine how to respond in the format.

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the output.
              If set to true, the model will always follow the exact schema defined
              in the `schema` field. Only a subset of JSON Schema is supported when
              `strict` is `true`. To learn more, read the [Structured Outputs
              guide](/api/docs/guides/structured-outputs).

          - `ResponseFormatJSONObject object { type }`

            JSON object response format. An older method of generating JSON responses.
            Using `json_schema` is recommended for models that support it. Note that the
            model will not generate JSON without a system or user message instructing it
            to do so.

      - `tools: optional array of Function { name, parameters, strict, 6 more }  or FileSearch { type, vector_store_ids, filters, 2 more }  or Computer { type }  or 13 more`

        An array of tools the model may call while generating a response. You
        can specify which tool to use by setting the `tool_choice` parameter.

        The two categories of tools you can provide the model are:

        - **Built-in tools**: Tools that are provided by OpenAI that extend the
          model's capabilities, like [web search](/api/docs/guides/tools-web-search)
          or [file search](/api/docs/guides/tools-file-search). Learn more about
          [built-in tools](/api/docs/guides/tools).
        - **Function calls (custom tools)**: Functions that are defined by you,
          enabling the model to call your own code. Learn more about
          [function calling](/api/docs/guides/function-calling).

        - `Function object { name, parameters, strict, 6 more }`

          Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling).

          - `name: string`

            The name of the function to call.

          - `parameters: map[unknown] or null`

            A JSON schema object describing the parameters of the function.

          - `strict: boolean or null`

            Whether strict parameter validation is enforced for this function tool.

          - `type: "function"`

            The type of the function tool. Always `function`.

            - `"function"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `async: optional boolean`

          - `defer_loading: optional boolean`

            Whether this function is deferred and loaded via tool search.

          - `description: optional string or null`

            A description of the function. Used by the model to determine whether or not to call the function.

          - `output_schema: optional map[unknown] or null`

            A JSON schema object describing the JSON value encoded in string outputs for this function.

        - `FileSearch object { type, vector_store_ids, filters, 2 more }`

          A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search).

          - `type: "file_search"`

            The type of the file search tool. Always `file_search`.

            - `"file_search"`

          - `vector_store_ids: array of string`

            The IDs of the vector stores to search.

          - `filters: optional ComparisonFilter or CompoundFilter or null`

            A filter to apply.

            - `ComparisonFilter object { key, type, value }`

              A filter used to compare a specified attribute key to a given value using a defined comparison operation.

              - `key: string`

                The key to compare against the value.

              - `type: "eq" or "ne" or "gt" or 5 more`

                Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`.

                - `eq`: equals
                - `ne`: not equal
                - `gt`: greater than
                - `gte`: greater than or equal
                - `lt`: less than
                - `lte`: less than or equal
                - `in`: in
                - `nin`: not in

                - `"eq"`

                - `"ne"`

                - `"gt"`

                - `"gte"`

                - `"lt"`

                - `"lte"`

                - `"in"`

                - `"nin"`

              - `value: string or number or boolean or array of string or number`

                The value to compare against the attribute key; supports string, number, or boolean types.

                - `string`

                - `number`

                - `boolean`

                - `array of string or number`

                  - `string`

                  - `number`

            - `CompoundFilter object { filters, type }`

              Combine multiple filters using `and` or `or`.

              - `filters: array of ComparisonFilter or CompoundFilter`

                Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.

                - `ComparisonFilter object { key, type, value }`

                  A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                - `CompoundFilter object { filters, type }`

                  Combine multiple filters using `and` or `or`.

              - `type: "and" or "or"`

                Type of operation: `and` or `or`.

                - `"and"`

                - `"or"`

          - `max_num_results: optional number`

            The maximum number of results to return. This number should be between 1 and 50 inclusive.

          - `ranking_options: optional object { hybrid_search, ranker, score_threshold }`

            Ranking options for search.

            - `hybrid_search: optional object { embedding_weight, text_weight }`

              Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

              - `embedding_weight: number`

                The weight of the embedding in the reciprocal ranking fusion.

              - `text_weight: number`

                The weight of the text in the reciprocal ranking fusion.

            - `ranker: optional "auto" or "default-2024-11-15"`

              The ranker to use for the file search.

              - `"auto"`

              - `"default-2024-11-15"`

            - `score_threshold: optional number`

              The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

        - `Computer object { type }`

          A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

          - `type: "computer"`

            The type of the computer tool. Always `computer`.

            - `"computer"`

        - `ComputerUsePreview object { display_height, display_width, environment, type }`

          A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

          - `display_height: number`

            The height of the computer display.

          - `display_width: number`

            The width of the computer display.

          - `environment: "windows" or "mac" or "linux" or 2 more`

            The type of computer environment to control.

            - `"windows"`

            - `"mac"`

            - `"linux"`

            - `"ubuntu"`

            - `"browser"`

          - `type: "computer_use_preview"`

            The type of the computer use tool. Always `computer_use_preview`.

            - `"computer_use_preview"`

        - `WebSearch object { type, external_web_access, filters, 2 more }`

          Search the Internet for sources related to the prompt. Learn more about the
          [web search tool](/api/docs/guides/tools-web-search).

          - `type: "web_search" or "web_search_2025_08_26"`

            The type of the web search tool. One of `web_search` or `web_search_2025_08_26`.

            - `"web_search"`

            - `"web_search_2025_08_26"`

          - `external_web_access: optional boolean`

            Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

          - `filters: optional object { allowed_domains }  or null`

            Filters for the search.

            - `allowed_domains: optional array of string or null`

              Allowed domains for the search. If not provided, all domains are allowed.
              Subdomains of the provided domains are allowed as well.

              Example: `["pubmed.ncbi.nlm.nih.gov"]`

          - `search_context_size: optional "low" or "medium" or "high"`

            High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

            - `"low"`

            - `"medium"`

            - `"high"`

          - `user_location: optional object { city, country, region, 2 more }  or null`

            The approximate location of the user. If omitted or null, defaults to the
            United States. To avoid this fallback, pass `{"type": "approximate"}` without
            location fields. To localize results, provide the relevant location fields.

            - `city: optional string or null`

              Free text input for the city of the user, e.g. `San Francisco`.

            - `country: optional string or null`

              The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

            - `region: optional string or null`

              Free text input for the region of the user, e.g. `California`.

            - `timezone: optional string or null`

              The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

            - `type: optional "approximate"`

              The type of location approximation. Always `approximate`.

              - `"approximate"`

        - `Mcp object { server_label, type, allowed_callers, 9 more }`

          Give the model access to additional tools via remote Model Context Protocol
          (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp).

          - `server_label: string`

            A label for this MCP server, used to identify it in tool calls.

          - `type: "mcp"`

            The type of the MCP tool. Always `mcp`.

            - `"mcp"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names }  or null`

            List of allowed tool names or a filter object.

            - `McpAllowedTools = array of string`

              A string array of allowed tool names

            - `McpToolFilter object { read_only, tool_names }`

              A filter object to specify which tools are allowed.

              - `read_only: optional boolean`

                Indicates whether or not a tool modifies data or is read-only. If an
                MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                it will match this filter.

              - `tool_names: optional array of string`

                List of allowed tool names.

          - `authorization: optional string`

            An OAuth access token that can be used with a remote MCP server, either
            with a custom MCP server URL or a service connector. Your application
            must handle the OAuth authorization flow and provide the token here.

          - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more`

            Identifier for service connectors, like those available in ChatGPT. One of
            `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more
            about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors).

            This field is deprecated for models released after September 1, 2026.
            Use `server_url` to connect to a remote MCP server, or `tunnel_id` to
            connect through a Secure MCP Tunnel.

            Currently supported `connector_id` values are:

            - Dropbox: `connector_dropbox`
            - Gmail: `connector_gmail`
            - Google Calendar: `connector_googlecalendar`
            - Google Drive: `connector_googledrive`
            - Microsoft Teams: `connector_microsoftteams`
            - Outlook Calendar: `connector_outlookcalendar`
            - Outlook Email: `connector_outlookemail`
            - SharePoint: `connector_sharepoint`

            - `"connector_dropbox"`

            - `"connector_gmail"`

            - `"connector_googlecalendar"`

            - `"connector_googledrive"`

            - `"connector_microsoftteams"`

            - `"connector_outlookcalendar"`

            - `"connector_outlookemail"`

            - `"connector_sharepoint"`

          - `defer_loading: optional boolean`

            Whether this MCP tool is deferred and discovered via tool search.

          - `headers: optional map[string] or null`

            Optional HTTP headers to send to the MCP server. Use for authentication
            or other purposes.

          - `require_approval: optional McpToolApprovalFilter { always, never }  or "always" or "never" or null`

            Specify which of the MCP server's tools require approval.

            - `McpToolApprovalFilter object { always, never }`

              Specify which of the MCP server's tools require approval. Can be
              `always`, `never`, or a filter object associated with tools
              that require approval.

              - `always: optional object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

              - `never: optional object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

            - `McpToolApprovalSetting = "always" or "never"`

              Specify a single approval policy for all tools. One of `always` or
              `never`. When set to `always`, all tools will require approval. When
              set to `never`, all tools will not require approval.

              - `"always"`

              - `"never"`

          - `server_description: optional string`

            Optional description of the MCP server, used to provide more context.

          - `server_url: optional string`

            The URL for the MCP server. One of `server_url`, `connector_id`, or
            `tunnel_id` must be provided.

          - `tunnel_id: optional string`

            The Secure MCP Tunnel ID to use instead of a direct server URL. One of
            `server_url`, `connector_id`, or `tunnel_id` must be provided.

        - `CodeInterpreter object { container, type, allowed_callers }`

          A tool that runs Python code to help generate a response to a prompt.

          - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }`

            The code interpreter container. Can be a container ID or an object that
            specifies uploaded file IDs to make available to your code, along with an
            optional `memory_limit` setting.

            - `string`

              The container ID.

            - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }`

              Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

              - `type: "auto"`

                Always `auto`.

                - `"auto"`

              - `file_ids: optional array of string`

                An optional list of uploaded files to make available to your code.

              - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                The memory limit for the code interpreter container.

                - `"1g"`

                - `"4g"`

                - `"16g"`

                - `"64g"`

              - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                Network access policy for the container.

                - `ContainerNetworkPolicyDisabled object { type }`

                  - `type: "disabled"`

                    Disable outbound network access. Always `disabled`.

                    - `"disabled"`

                - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                  - `allowed_domains: array of string`

                    A list of allowed domains when type is `allowlist`.

                  - `type: "allowlist"`

                    Allow outbound network access only to specified domains. Always `allowlist`.

                    - `"allowlist"`

                  - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret`

                    Optional domain-scoped secrets for allowlisted domains.

                    - `domain: string`

                      The domain associated with the secret.

                    - `name: string`

                      The name of the secret to inject for the domain.

                    - `value: string`

                      The secret value to inject for the domain.

          - `type: "code_interpreter"`

            The type of the code interpreter tool. Always `code_interpreter`.

            - `"code_interpreter"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

        - `ProgrammaticToolCalling object { type }`

          - `type: "programmatic_tool_calling"`

            The type of the tool. Always `programmatic_tool_calling`.

            - `"programmatic_tool_calling"`

        - `ImageGeneration object { type, action, background, 9 more }`

          A tool that generates images using the GPT image models.

          - `type: "image_generation"`

            The type of the image generation tool. Always `image_generation`.

            - `"image_generation"`

          - `action: optional "generate" or "edit" or "auto"`

            Whether to generate a new image or edit an existing image. Default: `auto`.

            - `"generate"`

            - `"edit"`

            - `"auto"`

          - `background: optional "transparent" or "opaque" or "auto"`

            Set the background of the generated image. One of `transparent`, `opaque`,
            or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including
            their `2026-09-08` snapshots, support `opaque` and `transparent`
            backgrounds. Transparent backgrounds are available for supported GPT Image
            models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in
            preview. When using `transparent`, set the output format to `png` or `webp`.
            Default: `auto`.

            - `"transparent"`

            - `"opaque"`

            - `"auto"`

          - `input_fidelity: optional "high" or "low" or null`

            Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models.

            - `"high"`

            - `"low"`

          - `input_image_mask: optional object { file_id, image_url }`

            Optional mask for inpainting. Contains `image_url`
            (string, optional) and `file_id` (string, optional).

            - `file_id: optional string`

              File ID for the mask image.

            - `image_url: optional string`

              Base64-encoded mask image.

          - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

            The image generation model to use. One of `gpt-image-1`,
            `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
            `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
            `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
            `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
            `gpt-image-1`.

            - `string`

            - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

              The image generation model to use. One of `gpt-image-1`,
              `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
              `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
              `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
              `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
              `gpt-image-1`.

              - `"gpt-image-1"`

              - `"gpt-image-1-mini"`

              - `"gpt-image-1.5"`

              - `"gpt-image-2"`

              - `"gpt-image-2-2026-04-21"`

              - `"gpt-image-2.5-sunburst"`

              - `"gpt-image-2.5-sunburst-2026-09-08"`

              - `"gpt-image-2.5-flare"`

              - `"gpt-image-2.5-flare-2026-09-08"`

          - `moderation: optional "auto" or "low"`

            Moderation level for the generated image. Default: `auto`.

            - `"auto"`

            - `"low"`

          - `output_compression: optional number`

            Compression level for the output image. Default: 100.

          - `output_format: optional "png" or "webp" or "jpeg"`

            The output format of the generated image. One of `png`, `webp`, or
            `jpeg`. Default: `png`.

            - `"png"`

            - `"webp"`

            - `"jpeg"`

          - `partial_images: optional number`

            Number of partial images to generate in streaming mode, from 0 (default value) to 3.

          - `quality: optional "low" or "medium" or "high" or 3 more`

            The quality of the generated image. The GPT image models support `low`,
            `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`,
            including their `2026-09-08` snapshots, also support `xhigh` and `max`.
            Default: `auto`.

            - `"low"`

            - `"medium"`

            - `"high"`

            - `"xhigh"`

            - `"max"`

            - `"auto"`

          - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"`

            The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

            - `string`

            - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"`

              The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

              - `"1024x1024"`

              - `"1024x1536"`

              - `"1536x1024"`

              - `"auto"`

        - `LocalShell object { type }`

          A tool that allows the model to execute shell commands in a local environment.

          - `type: "local_shell"`

            The type of the local shell tool. Always `local_shell`.

            - `"local_shell"`

        - `Shell object { type, allowed_callers, environment }`

          A tool that allows the model to execute shell commands.

          - `type: "shell"`

            The type of the shell tool. Always `shell`.

            - `"shell"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null`

            - `ContainerAuto object { type, file_ids, memory_limit, 2 more }`

              - `type: "container_auto"`

                Automatically creates a container for this request

                - `"container_auto"`

              - `file_ids: optional array of string`

                An optional list of uploaded files to make available to your code.

              - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                The memory limit for the container.

                - `"1g"`

                - `"4g"`

                - `"16g"`

                - `"64g"`

              - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                Network access policy for the container.

                - `ContainerNetworkPolicyDisabled object { type }`

                - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

              - `skills: optional array of SkillReference or InlineSkill`

                An optional list of skills referenced by id or inline data.

                - `SkillReference object { skill_id, type, version }`

                  - `skill_id: string`

                    The ID of the referenced skill.

                  - `type: "skill_reference"`

                    References a skill created with the /v1/skills endpoint.

                    - `"skill_reference"`

                  - `version: optional string`

                    Optional skill version. Use a positive integer or 'latest'. Omit for default.

                - `InlineSkill object { description, name, source, type }`

                  - `description: string`

                    The description of the skill.

                  - `name: string`

                    The name of the skill.

                  - `source: InlineSkillSource`

                    Inline skill payload

                    - `data: string`

                      Base64-encoded skill zip bundle.

                    - `media_type: "application/zip"`

                      The media type of the inline skill payload. Must be `application/zip`.

                      - `"application/zip"`

                    - `type: "base64"`

                      The type of the inline skill source. Must be `base64`.

                      - `"base64"`

                  - `type: "inline"`

                    Defines an inline skill for this request.

                    - `"inline"`

            - `LocalEnvironment object { type, skills }`

              - `type: "local"`

                Use a local computer environment.

                - `"local"`

              - `skills: optional array of LocalSkill`

                An optional list of skills.

                - `description: string`

                  The description of the skill.

                - `name: string`

                  The name of the skill.

                - `path: string`

                  The path to the directory containing the skill.

            - `ContainerReference object { container_id, type }`

              - `container_id: string`

                The ID of the referenced container.

              - `type: "container_reference"`

                References a container created with the /v1/containers endpoint

                - `"container_reference"`

        - `Custom object { name, type, allowed_callers, 4 more }`

          A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

          - `name: string`

            The name of the custom tool, used to identify it in tool calls.

          - `type: "custom"`

            The type of the custom tool. Always `custom`.

            - `"custom"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `async: optional boolean`

            Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

          - `defer_loading: optional boolean`

            Whether this tool should be deferred and discovered via tool search.

          - `description: optional string`

            Optional description of the custom tool, used to provide more context.

          - `format: optional CustomToolInputFormat`

            The input format for the custom tool. Default is unconstrained text.

            - `Text object { type }`

              Unconstrained free-form text.

              - `type: "text"`

                Unconstrained text format. Always `text`.

                - `"text"`

            - `Grammar object { definition, syntax, type }`

              A grammar defined by the user.

              - `definition: string`

                The grammar definition.

              - `syntax: "lark" or "regex"`

                The syntax of the grammar definition. One of `lark` or `regex`.

                - `"lark"`

                - `"regex"`

              - `type: "grammar"`

                Grammar format. Always `grammar`.

                - `"grammar"`

        - `Namespace object { description, name, tools, type }`

          Groups function/custom tools under a shared namespace.

          - `description: string`

            A description of the namespace shown to the model.

          - `name: string`

            The namespace name used in tool calls (for example, `crm`).

          - `tools: array of Function { name, type, allowed_callers, 6 more }  or Custom { name, type, allowed_callers, 4 more }`

            The function/custom tools available inside this namespace.

            - `Function object { name, type, allowed_callers, 6 more }`

              - `name: string`

              - `type: "function"`

                - `"function"`

              - `allowed_callers: optional array of "direct" or "programmatic" or null`

                The tool invocation context(s).

                - `"direct"`

                - `"programmatic"`

              - `async: optional boolean`

                Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

              - `defer_loading: optional boolean`

                Whether this function should be deferred and discovered via tool search.

              - `description: optional string or null`

              - `output_schema: optional map[unknown] or null`

                A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

              - `parameters: optional unknown or null`

              - `strict: optional boolean or null`

                Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

            - `Custom object { name, type, allowed_callers, 4 more }`

              A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

              - `name: string`

                The name of the custom tool, used to identify it in tool calls.

              - `type: "custom"`

                The type of the custom tool. Always `custom`.

                - `"custom"`

              - `allowed_callers: optional array of "direct" or "programmatic" or null`

                The tool invocation context(s).

                - `"direct"`

                - `"programmatic"`

              - `async: optional boolean`

                Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

              - `defer_loading: optional boolean`

                Whether this tool should be deferred and discovered via tool search.

              - `description: optional string`

                Optional description of the custom tool, used to provide more context.

              - `format: optional CustomToolInputFormat`

                The input format for the custom tool. Default is unconstrained text.

          - `type: "namespace"`

            The type of the tool. Always `namespace`.

            - `"namespace"`

        - `ToolSearch object { type, description, execution, parameters }`

          Hosted or BYOT tool search configuration for deferred tools.

          - `type: "tool_search"`

            The type of the tool. Always `tool_search`.

            - `"tool_search"`

          - `description: optional string or null`

            Description shown to the model for a client-executed tool search tool.

          - `execution: optional "server" or "client"`

            Whether tool search is executed by the server or by the client.

            - `"server"`

            - `"client"`

          - `parameters: optional unknown or null`

            Parameter schema for a client-executed tool search tool.

        - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }`

          This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search).

          - `type: "web_search_preview" or "web_search_preview_2025_03_11"`

            The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`.

            - `"web_search_preview"`

            - `"web_search_preview_2025_03_11"`

          - `search_content_types: optional array of "text" or "image"`

            - `"text"`

            - `"image"`

          - `search_context_size: optional "low" or "medium" or "high"`

            High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

            - `"low"`

            - `"medium"`

            - `"high"`

          - `user_location: optional object { type, city, country, 2 more }  or null`

            The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields.

            - `type: "approximate"`

              The type of location approximation. Always `approximate`.

              - `"approximate"`

            - `city: optional string or null`

              Free text input for the city of the user, e.g. `San Francisco`.

            - `country: optional string or null`

              The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

            - `region: optional string or null`

              Free text input for the region of the user, e.g. `California`.

            - `timezone: optional string or null`

              The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

        - `ApplyPatch object { type, allowed_callers }`

          Allows the assistant to create, delete, or update files using unified diffs.

          - `type: "apply_patch"`

            The type of the tool. Always `apply_patch`.

            - `"apply_patch"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

      - `top_p: optional number`

        An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

- `error: EvalAPIError or null`

  An object representing an error response from the Eval API.

  - `code: string`

    The error code.

  - `message: string`

    The error message.

- `eval_id: string`

  The identifier of the associated evaluation.

- `metadata: Metadata or null`

  Set of 16 key-value pairs that can be attached to an object. This can be
  useful for storing additional information about the object in a structured
  format, and querying for objects via API or the dashboard.

  Keys are strings with a maximum length of 64 characters. Values are strings
  with a maximum length of 512 characters.

- `model: string or null`

  The model that is evaluated, if applicable.

- `name: string or null`

  The name of the evaluation run.

- `object: "eval.run"`

  The type of the object. Always "eval.run".

  - `"eval.run"`

- `per_model_usage: array of object { cached_tokens, completion_tokens, invocation_count, 3 more }  or null`

  Usage statistics for each model during the evaluation run.

  - `cached_tokens: number`

    The number of tokens retrieved from cache.

  - `completion_tokens: number`

    The number of completion tokens generated.

  - `invocation_count: number`

    The number of invocations.

  - `model_name: string`

    The name of the model.

  - `prompt_tokens: number`

    The number of prompt tokens used.

  - `total_tokens: number`

    The total number of tokens used.

- `per_testing_criteria_results: array of object { failed, passed, testing_criteria }  or null`

  Results per testing criteria applied during the evaluation run.

  - `failed: number`

    Number of tests failed for this criteria.

  - `passed: number`

    Number of tests passed for this criteria.

  - `testing_criteria: string`

    A description of the testing criteria.

- `report_url: string`

  The URL to the rendered evaluation run report on the UI dashboard.

- `result_counts: object { errored, failed, passed, total }`

  Counters summarizing the outcomes of the evaluation run.

  - `errored: number`

    Number of output items that resulted in an error.

  - `failed: number`

    Number of output items that failed to pass the evaluation.

  - `passed: number`

    Number of output items that passed the evaluation.

  - `total: number`

    Total number of executed output items.

- `status: string`

  The status of the evaluation run.

### Example

```http
curl https://api.openai.com/v1/evals/$EVAL_ID/runs/$RUN_ID/cancel \
    -X POST \
    -H "Authorization: Bearer $OPENAI_API_KEY"
```

#### Response

```json
{
  "id": "id",
  "created_at": 0,
  "data_source": {
    "source": {
      "content": [
        {
          "item": {
            "foo": "bar"
          },
          "sample": {
            "foo": "bar"
          }
        }
      ],
      "type": "file_content"
    },
    "type": "jsonl"
  },
  "error": {
    "code": "code",
    "message": "message"
  },
  "eval_id": "eval_id",
  "metadata": {
    "foo": "string"
  },
  "model": "model",
  "name": "name",
  "object": "eval.run",
  "per_model_usage": [
    {
      "cached_tokens": 0,
      "completion_tokens": 0,
      "invocation_count": 0,
      "model_name": "model_name",
      "prompt_tokens": 0,
      "total_tokens": 0
    }
  ],
  "per_testing_criteria_results": [
    {
      "failed": 0,
      "passed": 0,
      "testing_criteria": "testing_criteria"
    }
  ],
  "report_url": "https://example.com",
  "result_counts": {
    "errored": 0,
    "failed": 0,
    "passed": 0,
    "total": 0
  },
  "status": "status"
}
```

### Example

```http
curl https://api.openai.com/v1/evals/eval_67abd54d9b0081909a86353f6fb9317a/runs/evalrun_67abd54d60ec8190832b46859da808f7/cancel \
  -X POST \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json"
```

#### Response

```json
{
  "object": "eval.run",
  "id": "evalrun_67abd54d60ec8190832b46859da808f7",
  "eval_id": "eval_67abd54d9b0081909a86353f6fb9317a",
  "report_url": "https://platform.openai.com/evaluations/eval_67abd54d9b0081909a86353f6fb9317a?run_id=evalrun_67abd54d60ec8190832b46859da808f7",
  "status": "canceled",
  "model": "gpt-6-astra",
  "name": "gpt-6-astra",
  "created_at": 1743092069,
  "result_counts": {
    "total": 0,
    "errored": 0,
    "failed": 0,
    "passed": 0
  },
  "per_model_usage": null,
  "per_testing_criteria_results": null,
  "data_source": {
    "type": "completions",
    "source": {
      "type": "file_content",
      "content": [
        {
          "item": {
            "input": "Tech Company Launches Advanced Artificial Intelligence Platform",
            "ground_truth": "Technology"
          }
        },
        {
          "item": {
            "input": "Central Bank Increases Interest Rates Amid Inflation Concerns",
            "ground_truth": "Markets"
          }
        },
        {
          "item": {
            "input": "International Summit Addresses Climate Change Strategies",
            "ground_truth": "World"
          }
        },
        {
          "item": {
            "input": "Major Retailer Reports Record-Breaking Holiday Sales",
            "ground_truth": "Business"
          }
        },
        {
          "item": {
            "input": "National Team Qualifies for World Championship Finals",
            "ground_truth": "Sports"
          }
        },
        {
          "item": {
            "input": "Stock Markets Rally After Positive Economic Data Released",
            "ground_truth": "Markets"
          }
        },
        {
          "item": {
            "input": "Global Manufacturer Announces Merger with Competitor",
            "ground_truth": "Business"
          }
        },
        {
          "item": {
            "input": "Breakthrough in Renewable Energy Technology Unveiled",
            "ground_truth": "Technology"
          }
        },
        {
          "item": {
            "input": "World Leaders Sign Historic Climate Agreement",
            "ground_truth": "World"
          }
        },
        {
          "item": {
            "input": "Professional Athlete Sets New Record in Championship Event",
            "ground_truth": "Sports"
          }
        },
        {
          "item": {
            "input": "Financial Institutions Adapt to New Regulatory Requirements",
            "ground_truth": "Business"
          }
        },
        {
          "item": {
            "input": "Tech Conference Showcases Advances in Artificial Intelligence",
            "ground_truth": "Technology"
          }
        },
        {
          "item": {
            "input": "Global Markets Respond to Oil Price Fluctuations",
            "ground_truth": "Markets"
          }
        },
        {
          "item": {
            "input": "International Cooperation Strengthened Through New Treaty",
            "ground_truth": "World"
          }
        },
        {
          "item": {
            "input": "Sports League Announces Revised Schedule for Upcoming Season",
            "ground_truth": "Sports"
          }
        }
      ]
    },
    "input_messages": {
      "type": "template",
      "template": [
        {
          "type": "message",
          "role": "developer",
          "content": {
            "type": "input_text",
            "text": "Categorize a given news headline into one of the following topics: Technology, Markets, World, Business, or Sports.\n\n# Steps\n\n1. Analyze the content of the news headline to understand its primary focus.\n2. Extract the subject matter, identifying any key indicators or keywords.\n3. Use the identified indicators to determine the most suitable category out of the five options: Technology, Markets, World, Business, or Sports.\n4. Ensure only one category is selected per headline.\n\n# Output Format\n\nRespond with the chosen category as a single word. For instance: \"Technology\", \"Markets\", \"World\", \"Business\", or \"Sports\".\n\n# Examples\n\n**Input**: \"Apple Unveils New iPhone Model, Featuring Advanced AI Features\"  \n**Output**: \"Technology\"\n\n**Input**: \"Global Stocks Mixed as Investors Await Central Bank Decisions\"  \n**Output**: \"Markets\"\n\n**Input**: \"War in Ukraine: Latest Updates on Negotiation Status\"  \n**Output**: \"World\"\n\n**Input**: \"Microsoft in Talks to Acquire Gaming Company for $2 Billion\"  \n**Output**: \"Business\"\n\n**Input**: \"Manchester United Secures Win in Premier League Football Match\"  \n**Output**: \"Sports\" \n\n# Notes\n\n- If the headline appears to fit into more than one category, choose the most dominant theme.\n- Keywords or phrases such as \"stocks\", \"company acquisition\", \"match\", or technological brands can be good indicators for classification.\n"
          }
        },
        {
          "type": "message",
          "role": "user",
          "content": {
            "type": "input_text",
            "text": "{{item.input}}"
          }
        }
      ]
    },
    "model": "gpt-6-astra",
    "sampling_params": {
      "max_completions_tokens": 2048
    }
  },
  "error": null,
  "metadata": {}
}
```

## Create eval run

**post** `/evals/{eval_id}/runs`

Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the evaluation.

### Path Parameters

- `eval_id: string`

### Body Parameters

- `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }`

  Details about the run's data source.

  - `CreateEvalJSONLRunDataSource object { source, type }`

    A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

      Determines what populates the `item` namespace in the data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

    - `type: "jsonl"`

      The type of data source. Always `jsonl`.

      - `"jsonl"`

  - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

    A CompletionsRunDataSource object describing a model sampling configuration.

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

      Determines what populates the `item` namespace in this run's data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

      - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

        A StoredCompletionsRunDataSource configuration describing a set of filters

        - `type: "stored_completions"`

          The type of source. Always `stored_completions`.

          - `"stored_completions"`

        - `created_after: optional number or null`

          An optional Unix timestamp to filter items created after this time.

        - `created_before: optional number or null`

          An optional Unix timestamp to filter items created before this time.

        - `limit: optional number or null`

          An optional maximum number of items to return.

        - `metadata: optional Metadata or null`

          Set of 16 key-value pairs that can be attached to an object. This can be
          useful for storing additional information about the object in a structured
          format, and querying for objects via API or the dashboard.

          Keys are strings with a maximum length of 64 characters. Values are strings
          with a maximum length of 512 characters.

        - `model: optional string or null`

          An optional model to filter by (e.g., 'gpt-6-astra').

    - `type: "completions"`

      The type of run data source. Always `completions`.

      - `"completions"`

    - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

      Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

      - `TemplateInputMessages object { template, type }`

        - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

          A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

          - `EasyInputMessage object { content, role, phase, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputMessageContentList`

              Text, image, or audio input to the model, used to generate a response.
              Can also contain previous assistant responses.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputMessageContentList = array of ResponseInputContent`

                A list of one or many input items to the model, containing different content
                types.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                  - `text: string`

                    The text input to the model.

                  - `type: "input_text"`

                    The type of the input item. Always `input_text`.

                    - `"input_text"`

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

                - `ResponseInputImage object { detail, type, file_id, 2 more }`

                  An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                  - `detail: ImageDetail`

                    The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                    - `"low"`

                    - `"high"`

                    - `"auto"`

                    - `"original"`

                  - `type: "input_image"`

                    The type of the input item. Always `input_image`.

                    - `"input_image"`

                  - `file_id: optional string or null`

                    The ID of the file to be sent to the model.

                  - `image_url: optional string or null`

                    The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

                - `ResponseInputFile object { type, detail, file_data, 4 more }`

                  A file input to the model.

                  - `type: "input_file"`

                    The type of the input item. Always `input_file`.

                    - `"input_file"`

                  - `detail: optional "auto" or "low" or "high"`

                    The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                    - `"auto"`

                    - `"low"`

                    - `"high"`

                  - `file_data: optional string`

                    The content of the file to be sent to the model.

                  - `file_id: optional string or null`

                    The ID of the file to be sent to the model.

                  - `file_url: optional string`

                    The URL of the file to be sent to the model.

                  - `filename: optional string`

                    The name of the file to be sent to the model.

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `phase: optional "commentary" or "final_answer" or null`

              Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
              For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
              phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

              - `"commentary"`

              - `"final_answer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

          - `EvalMessageObject object { content, role, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

              Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

              - `OutputText object { text, type }`

                A text output from the model.

                - `text: string`

                  The text output from the model.

                - `type: "output_text"`

                  The type of the output text. Always `output_text`.

                  - `"output_text"`

              - `InputImage object { image_url, type, detail }`

                An image input block used within EvalItem content arrays.

                - `image_url: string`

                  The URL of the image input.

                - `type: "input_image"`

                  The type of the image input. Always `input_image`.

                  - `"input_image"`

                - `detail: optional string`

                  The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

              - `ResponseInputAudio object { input_audio, type }`

                An audio input to the model.

                - `input_audio: object { data, format }`

                  - `data: string`

                    Base64-encoded audio data.

                  - `format: "mp3" or "wav"`

                    The format of the audio data. Currently supported formats are `mp3` and
                    `wav`.

                    - `"mp3"`

                    - `"wav"`

                - `type: "input_audio"`

                  The type of the input item. Always `input_audio`.

                  - `"input_audio"`

              - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                A list of inputs, each of which may be either an input text, output text, input
                image, or input audio object.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

        - `type: "template"`

          The type of input messages. Always `template`.

          - `"template"`

      - `ItemReferenceInputMessages object { item_reference, type }`

        - `item_reference: string`

          A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

        - `type: "item_reference"`

          The type of input messages. Always `item_reference`.

          - `"item_reference"`

    - `model: optional string`

      The name of the model to use for generating completions (e.g. "o3-mini").

    - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

      - `max_completion_tokens: optional number`

        The maximum number of tokens in the generated output.

      - `reasoning_effort: optional ReasoningEffort or null`

        Constrains effort on reasoning for reasoning models. Currently supported
        values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
        Reducing reasoning effort can result in faster responses and fewer tokens
        used on reasoning in a response. Not all reasoning models support every
        value. See the
        [reasoning guide](/api/docs/guides/reasoning)
        for model-specific support.

        - `"none"`

        - `"minimal"`

        - `"low"`

        - `"medium"`

        - `"high"`

        - `"xhigh"`

        - `"max"`

      - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

        An object specifying the format that the model must output.

        Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
        Structured Outputs which ensures the model will match your supplied JSON
        schema. Learn more in the [Structured Outputs
        guide](/api/docs/guides/structured-outputs).

        Setting to `{ "type": "json_object" }` enables the older JSON mode, which
        ensures the message the model generates is valid JSON. Using `json_schema`
        is preferred for models that support it.

        - `ResponseFormatText object { type }`

          Default response format. Used to generate text responses.

          - `type: "text"`

            The type of response format being defined. Always `text`.

            - `"text"`

        - `ResponseFormatJSONSchema object { json_schema, type }`

          JSON Schema response format. Used to generate structured JSON responses.
          Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

          - `json_schema: object { name, description, schema, strict }`

            Structured Outputs configuration options, including a JSON Schema.

            - `name: string`

              The name of the response format. Must be a-z, A-Z, 0-9, or contain
              underscores and dashes, with a maximum length of 64.

            - `description: optional string`

              A description of what the response format is for, used by the model to
              determine how to respond in the format.

            - `schema: optional map[unknown]`

              The schema for the response format, described as a JSON Schema object.
              Learn how to build JSON schemas [here](https://json-schema.org/).

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the output.
              If set to true, the model will always follow the exact schema defined
              in the `schema` field. Only a subset of JSON Schema is supported when
              `strict` is `true`. To learn more, read the [Structured Outputs
              guide](/api/docs/guides/structured-outputs).

          - `type: "json_schema"`

            The type of response format being defined. Always `json_schema`.

            - `"json_schema"`

        - `ResponseFormatJSONObject object { type }`

          JSON object response format. An older method of generating JSON responses.
          Using `json_schema` is recommended for models that support it. Note that the
          model will not generate JSON without a system or user message instructing it
          to do so.

          - `type: "json_object"`

            The type of response format being defined. Always `json_object`.

            - `"json_object"`

      - `seed: optional number`

        A seed value to initialize the randomness, during sampling.

      - `temperature: optional number`

        A higher temperature increases randomness in the outputs.

      - `tools: optional array of ChatCompletionFunctionTool`

        A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

        - `function: FunctionDefinition`

          - `name: string`

            The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

          - `description: optional string`

            A description of what the function does, used by the model to choose when and how to call the function.

          - `parameters: optional FunctionParameters`

            The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

            Omitting `parameters` defines a function with an empty parameter list.

          - `strict: optional boolean or null`

            Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

        - `type: "function"`

          The type of the tool. Currently, only `function` is supported.

          - `"function"`

      - `top_p: optional number`

        An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

  - `ResponsesRunDataSource object { source, type, input_messages, 2 more }`

    A ResponsesRunDataSource object describing a model sampling configuration.

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or EvalResponsesSource { type, created_after, created_before, 8 more }`

      Determines what populates the `item` namespace in this run's data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

      - `EvalResponsesSource object { type, created_after, created_before, 8 more }`

        A EvalResponsesSource object describing a run data source configuration.

        - `type: "responses"`

          The type of run data source. Always `responses`.

          - `"responses"`

        - `created_after: optional number or null`

          Only include items created after this timestamp (inclusive). This is a query parameter used to select responses.

        - `created_before: optional number or null`

          Only include items created before this timestamp (inclusive). This is a query parameter used to select responses.

        - `instructions_search: optional string or null`

          Optional string to search the 'instructions' field. This is a query parameter used to select responses.

        - `metadata: optional unknown or null`

          Metadata filter for the responses. This is a query parameter used to select responses.

        - `model: optional string or null`

          The name of the model to find responses for. This is a query parameter used to select responses.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

        - `temperature: optional number or null`

          Sampling temperature. This is a query parameter used to select responses.

        - `tools: optional array of string or null`

          List of tool names. This is a query parameter used to select responses.

        - `top_p: optional number or null`

          Nucleus sampling parameter. This is a query parameter used to select responses.

        - `users: optional array of string or null`

          List of user identifiers. This is a query parameter used to select responses.

    - `type: "responses"`

      The type of run data source. Always `responses`.

      - `"responses"`

    - `input_messages: optional InputMessagesTemplate { template, type }  or InputMessagesItemReference { item_reference, type }`

      Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

      - `InputMessagesTemplate object { template, type }`

        - `template: array of ChatMessage { content, role }  or EvalMessageObject { content, role, type }`

          A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

          - `ChatMessage object { content, role }`

            - `content: string`

              The content of the message.

            - `role: string`

              The role of the message (e.g. "system", "assistant", "user").

          - `EvalMessageObject object { content, role, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

              Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

              - `OutputText object { text, type }`

                A text output from the model.

                - `text: string`

                  The text output from the model.

                - `type: "output_text"`

                  The type of the output text. Always `output_text`.

                  - `"output_text"`

              - `InputImage object { image_url, type, detail }`

                An image input block used within EvalItem content arrays.

                - `image_url: string`

                  The URL of the image input.

                - `type: "input_image"`

                  The type of the image input. Always `input_image`.

                  - `"input_image"`

                - `detail: optional string`

                  The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

              - `ResponseInputAudio object { input_audio, type }`

                An audio input to the model.

              - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                A list of inputs, each of which may be either an input text, output text, input
                image, or input audio object.

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

        - `type: "template"`

          The type of input messages. Always `template`.

          - `"template"`

      - `InputMessagesItemReference object { item_reference, type }`

        - `item_reference: string`

          A reference to a variable in the `item` namespace. Ie, "item.name"

        - `type: "item_reference"`

          The type of input messages. Always `item_reference`.

          - `"item_reference"`

    - `model: optional string`

      The name of the model to use for generating completions (e.g. "o3-mini").

    - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }`

      - `max_completion_tokens: optional number`

        The maximum number of tokens in the generated output.

      - `reasoning_effort: optional ReasoningEffort or null`

        Constrains effort on reasoning for reasoning models. Currently supported
        values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
        Reducing reasoning effort can result in faster responses and fewer tokens
        used on reasoning in a response. Not all reasoning models support every
        value. See the
        [reasoning guide](/api/docs/guides/reasoning)
        for model-specific support.

      - `seed: optional number`

        A seed value to initialize the randomness, during sampling.

      - `temperature: optional number`

        A higher temperature increases randomness in the outputs.

      - `text: optional object { format }`

        Configuration options for a text response from the model. Can be plain
        text or structured JSON data. Learn more:

        - [Text inputs and outputs](/api/docs/guides/text)
        - [Structured Outputs](/api/docs/guides/structured-outputs)

        - `format: optional ResponseFormatTextConfig`

          An object specifying the format that the model must output.

          Configuring `{ "type": "json_schema" }` enables Structured Outputs,
          which ensures the model will match your supplied JSON schema. Learn more in the
          [Structured Outputs guide](/api/docs/guides/structured-outputs).

          The default format is `{ "type": "text" }` with no additional options.

          **Not recommended for gpt-4o and newer models:**

          Setting to `{ "type": "json_object" }` enables the older JSON mode, which
          ensures the message the model generates is valid JSON. Using `json_schema`
          is preferred for models that support it.

          - `ResponseFormatText object { type }`

            Default response format. Used to generate text responses.

          - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }`

            JSON Schema response format. Used to generate structured JSON responses.
            Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

            - `name: string`

              The name of the response format. Must be a-z, A-Z, 0-9, or contain
              underscores and dashes, with a maximum length of 64.

            - `schema: map[unknown]`

              The schema for the response format, described as a JSON Schema object.
              Learn how to build JSON schemas [here](https://json-schema.org/).

            - `type: "json_schema"`

              The type of response format being defined. Always `json_schema`.

              - `"json_schema"`

            - `description: optional string`

              A description of what the response format is for, used by the model to
              determine how to respond in the format.

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the output.
              If set to true, the model will always follow the exact schema defined
              in the `schema` field. Only a subset of JSON Schema is supported when
              `strict` is `true`. To learn more, read the [Structured Outputs
              guide](/api/docs/guides/structured-outputs).

          - `ResponseFormatJSONObject object { type }`

            JSON object response format. An older method of generating JSON responses.
            Using `json_schema` is recommended for models that support it. Note that the
            model will not generate JSON without a system or user message instructing it
            to do so.

      - `tools: optional array of Function { name, parameters, strict, 6 more }  or FileSearch { type, vector_store_ids, filters, 2 more }  or Computer { type }  or 13 more`

        An array of tools the model may call while generating a response. You
        can specify which tool to use by setting the `tool_choice` parameter.

        The two categories of tools you can provide the model are:

        - **Built-in tools**: Tools that are provided by OpenAI that extend the
          model's capabilities, like [web search](/api/docs/guides/tools-web-search)
          or [file search](/api/docs/guides/tools-file-search). Learn more about
          [built-in tools](/api/docs/guides/tools).
        - **Function calls (custom tools)**: Functions that are defined by you,
          enabling the model to call your own code. Learn more about
          [function calling](/api/docs/guides/function-calling).

        - `Function object { name, parameters, strict, 6 more }`

          Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling).

          - `name: string`

            The name of the function to call.

          - `parameters: map[unknown] or null`

            A JSON schema object describing the parameters of the function.

          - `strict: boolean or null`

            Whether strict parameter validation is enforced for this function tool.

          - `type: "function"`

            The type of the function tool. Always `function`.

            - `"function"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `async: optional boolean`

          - `defer_loading: optional boolean`

            Whether this function is deferred and loaded via tool search.

          - `description: optional string or null`

            A description of the function. Used by the model to determine whether or not to call the function.

          - `output_schema: optional map[unknown] or null`

            A JSON schema object describing the JSON value encoded in string outputs for this function.

        - `FileSearch object { type, vector_store_ids, filters, 2 more }`

          A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search).

          - `type: "file_search"`

            The type of the file search tool. Always `file_search`.

            - `"file_search"`

          - `vector_store_ids: array of string`

            The IDs of the vector stores to search.

          - `filters: optional ComparisonFilter or CompoundFilter or null`

            A filter to apply.

            - `ComparisonFilter object { key, type, value }`

              A filter used to compare a specified attribute key to a given value using a defined comparison operation.

              - `key: string`

                The key to compare against the value.

              - `type: "eq" or "ne" or "gt" or 5 more`

                Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`.

                - `eq`: equals
                - `ne`: not equal
                - `gt`: greater than
                - `gte`: greater than or equal
                - `lt`: less than
                - `lte`: less than or equal
                - `in`: in
                - `nin`: not in

                - `"eq"`

                - `"ne"`

                - `"gt"`

                - `"gte"`

                - `"lt"`

                - `"lte"`

                - `"in"`

                - `"nin"`

              - `value: string or number or boolean or array of string or number`

                The value to compare against the attribute key; supports string, number, or boolean types.

                - `string`

                - `number`

                - `boolean`

                - `array of string or number`

                  - `string`

                  - `number`

            - `CompoundFilter object { filters, type }`

              Combine multiple filters using `and` or `or`.

              - `filters: array of ComparisonFilter or CompoundFilter`

                Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.

                - `ComparisonFilter object { key, type, value }`

                  A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                - `CompoundFilter object { filters, type }`

                  Combine multiple filters using `and` or `or`.

              - `type: "and" or "or"`

                Type of operation: `and` or `or`.

                - `"and"`

                - `"or"`

          - `max_num_results: optional number`

            The maximum number of results to return. This number should be between 1 and 50 inclusive.

          - `ranking_options: optional object { hybrid_search, ranker, score_threshold }`

            Ranking options for search.

            - `hybrid_search: optional object { embedding_weight, text_weight }`

              Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

              - `embedding_weight: number`

                The weight of the embedding in the reciprocal ranking fusion.

              - `text_weight: number`

                The weight of the text in the reciprocal ranking fusion.

            - `ranker: optional "auto" or "default-2024-11-15"`

              The ranker to use for the file search.

              - `"auto"`

              - `"default-2024-11-15"`

            - `score_threshold: optional number`

              The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

        - `Computer object { type }`

          A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

          - `type: "computer"`

            The type of the computer tool. Always `computer`.

            - `"computer"`

        - `ComputerUsePreview object { display_height, display_width, environment, type }`

          A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

          - `display_height: number`

            The height of the computer display.

          - `display_width: number`

            The width of the computer display.

          - `environment: "windows" or "mac" or "linux" or 2 more`

            The type of computer environment to control.

            - `"windows"`

            - `"mac"`

            - `"linux"`

            - `"ubuntu"`

            - `"browser"`

          - `type: "computer_use_preview"`

            The type of the computer use tool. Always `computer_use_preview`.

            - `"computer_use_preview"`

        - `WebSearch object { type, external_web_access, filters, 2 more }`

          Search the Internet for sources related to the prompt. Learn more about the
          [web search tool](/api/docs/guides/tools-web-search).

          - `type: "web_search" or "web_search_2025_08_26"`

            The type of the web search tool. One of `web_search` or `web_search_2025_08_26`.

            - `"web_search"`

            - `"web_search_2025_08_26"`

          - `external_web_access: optional boolean`

            Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

          - `filters: optional object { allowed_domains }  or null`

            Filters for the search.

            - `allowed_domains: optional array of string or null`

              Allowed domains for the search. If not provided, all domains are allowed.
              Subdomains of the provided domains are allowed as well.

              Example: `["pubmed.ncbi.nlm.nih.gov"]`

          - `search_context_size: optional "low" or "medium" or "high"`

            High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

            - `"low"`

            - `"medium"`

            - `"high"`

          - `user_location: optional object { city, country, region, 2 more }  or null`

            The approximate location of the user. If omitted or null, defaults to the
            United States. To avoid this fallback, pass `{"type": "approximate"}` without
            location fields. To localize results, provide the relevant location fields.

            - `city: optional string or null`

              Free text input for the city of the user, e.g. `San Francisco`.

            - `country: optional string or null`

              The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

            - `region: optional string or null`

              Free text input for the region of the user, e.g. `California`.

            - `timezone: optional string or null`

              The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

            - `type: optional "approximate"`

              The type of location approximation. Always `approximate`.

              - `"approximate"`

        - `Mcp object { server_label, type, allowed_callers, 9 more }`

          Give the model access to additional tools via remote Model Context Protocol
          (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp).

          - `server_label: string`

            A label for this MCP server, used to identify it in tool calls.

          - `type: "mcp"`

            The type of the MCP tool. Always `mcp`.

            - `"mcp"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names }  or null`

            List of allowed tool names or a filter object.

            - `McpAllowedTools = array of string`

              A string array of allowed tool names

            - `McpToolFilter object { read_only, tool_names }`

              A filter object to specify which tools are allowed.

              - `read_only: optional boolean`

                Indicates whether or not a tool modifies data or is read-only. If an
                MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                it will match this filter.

              - `tool_names: optional array of string`

                List of allowed tool names.

          - `authorization: optional string`

            An OAuth access token that can be used with a remote MCP server, either
            with a custom MCP server URL or a service connector. Your application
            must handle the OAuth authorization flow and provide the token here.

          - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more`

            Identifier for service connectors, like those available in ChatGPT. One of
            `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more
            about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors).

            This field is deprecated for models released after September 1, 2026.
            Use `server_url` to connect to a remote MCP server, or `tunnel_id` to
            connect through a Secure MCP Tunnel.

            Currently supported `connector_id` values are:

            - Dropbox: `connector_dropbox`
            - Gmail: `connector_gmail`
            - Google Calendar: `connector_googlecalendar`
            - Google Drive: `connector_googledrive`
            - Microsoft Teams: `connector_microsoftteams`
            - Outlook Calendar: `connector_outlookcalendar`
            - Outlook Email: `connector_outlookemail`
            - SharePoint: `connector_sharepoint`

            - `"connector_dropbox"`

            - `"connector_gmail"`

            - `"connector_googlecalendar"`

            - `"connector_googledrive"`

            - `"connector_microsoftteams"`

            - `"connector_outlookcalendar"`

            - `"connector_outlookemail"`

            - `"connector_sharepoint"`

          - `defer_loading: optional boolean`

            Whether this MCP tool is deferred and discovered via tool search.

          - `headers: optional map[string] or null`

            Optional HTTP headers to send to the MCP server. Use for authentication
            or other purposes.

          - `require_approval: optional McpToolApprovalFilter { always, never }  or "always" or "never" or null`

            Specify which of the MCP server's tools require approval.

            - `McpToolApprovalFilter object { always, never }`

              Specify which of the MCP server's tools require approval. Can be
              `always`, `never`, or a filter object associated with tools
              that require approval.

              - `always: optional object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

              - `never: optional object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

            - `McpToolApprovalSetting = "always" or "never"`

              Specify a single approval policy for all tools. One of `always` or
              `never`. When set to `always`, all tools will require approval. When
              set to `never`, all tools will not require approval.

              - `"always"`

              - `"never"`

          - `server_description: optional string`

            Optional description of the MCP server, used to provide more context.

          - `server_url: optional string`

            The URL for the MCP server. One of `server_url`, `connector_id`, or
            `tunnel_id` must be provided.

          - `tunnel_id: optional string`

            The Secure MCP Tunnel ID to use instead of a direct server URL. One of
            `server_url`, `connector_id`, or `tunnel_id` must be provided.

        - `CodeInterpreter object { container, type, allowed_callers }`

          A tool that runs Python code to help generate a response to a prompt.

          - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }`

            The code interpreter container. Can be a container ID or an object that
            specifies uploaded file IDs to make available to your code, along with an
            optional `memory_limit` setting.

            - `string`

              The container ID.

            - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }`

              Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

              - `type: "auto"`

                Always `auto`.

                - `"auto"`

              - `file_ids: optional array of string`

                An optional list of uploaded files to make available to your code.

              - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                The memory limit for the code interpreter container.

                - `"1g"`

                - `"4g"`

                - `"16g"`

                - `"64g"`

              - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                Network access policy for the container.

                - `ContainerNetworkPolicyDisabled object { type }`

                  - `type: "disabled"`

                    Disable outbound network access. Always `disabled`.

                    - `"disabled"`

                - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                  - `allowed_domains: array of string`

                    A list of allowed domains when type is `allowlist`.

                  - `type: "allowlist"`

                    Allow outbound network access only to specified domains. Always `allowlist`.

                    - `"allowlist"`

                  - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret`

                    Optional domain-scoped secrets for allowlisted domains.

                    - `domain: string`

                      The domain associated with the secret.

                    - `name: string`

                      The name of the secret to inject for the domain.

                    - `value: string`

                      The secret value to inject for the domain.

          - `type: "code_interpreter"`

            The type of the code interpreter tool. Always `code_interpreter`.

            - `"code_interpreter"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

        - `ProgrammaticToolCalling object { type }`

          - `type: "programmatic_tool_calling"`

            The type of the tool. Always `programmatic_tool_calling`.

            - `"programmatic_tool_calling"`

        - `ImageGeneration object { type, action, background, 9 more }`

          A tool that generates images using the GPT image models.

          - `type: "image_generation"`

            The type of the image generation tool. Always `image_generation`.

            - `"image_generation"`

          - `action: optional "generate" or "edit" or "auto"`

            Whether to generate a new image or edit an existing image. Default: `auto`.

            - `"generate"`

            - `"edit"`

            - `"auto"`

          - `background: optional "transparent" or "opaque" or "auto"`

            Set the background of the generated image. One of `transparent`, `opaque`,
            or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including
            their `2026-09-08` snapshots, support `opaque` and `transparent`
            backgrounds. Transparent backgrounds are available for supported GPT Image
            models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in
            preview. When using `transparent`, set the output format to `png` or `webp`.
            Default: `auto`.

            - `"transparent"`

            - `"opaque"`

            - `"auto"`

          - `input_fidelity: optional "high" or "low" or null`

            Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models.

            - `"high"`

            - `"low"`

          - `input_image_mask: optional object { file_id, image_url }`

            Optional mask for inpainting. Contains `image_url`
            (string, optional) and `file_id` (string, optional).

            - `file_id: optional string`

              File ID for the mask image.

            - `image_url: optional string`

              Base64-encoded mask image.

          - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

            The image generation model to use. One of `gpt-image-1`,
            `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
            `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
            `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
            `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
            `gpt-image-1`.

            - `string`

            - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

              The image generation model to use. One of `gpt-image-1`,
              `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
              `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
              `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
              `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
              `gpt-image-1`.

              - `"gpt-image-1"`

              - `"gpt-image-1-mini"`

              - `"gpt-image-1.5"`

              - `"gpt-image-2"`

              - `"gpt-image-2-2026-04-21"`

              - `"gpt-image-2.5-sunburst"`

              - `"gpt-image-2.5-sunburst-2026-09-08"`

              - `"gpt-image-2.5-flare"`

              - `"gpt-image-2.5-flare-2026-09-08"`

          - `moderation: optional "auto" or "low"`

            Moderation level for the generated image. Default: `auto`.

            - `"auto"`

            - `"low"`

          - `output_compression: optional number`

            Compression level for the output image. Default: 100.

          - `output_format: optional "png" or "webp" or "jpeg"`

            The output format of the generated image. One of `png`, `webp`, or
            `jpeg`. Default: `png`.

            - `"png"`

            - `"webp"`

            - `"jpeg"`

          - `partial_images: optional number`

            Number of partial images to generate in streaming mode, from 0 (default value) to 3.

          - `quality: optional "low" or "medium" or "high" or 3 more`

            The quality of the generated image. The GPT image models support `low`,
            `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`,
            including their `2026-09-08` snapshots, also support `xhigh` and `max`.
            Default: `auto`.

            - `"low"`

            - `"medium"`

            - `"high"`

            - `"xhigh"`

            - `"max"`

            - `"auto"`

          - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"`

            The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

            - `string`

            - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"`

              The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

              - `"1024x1024"`

              - `"1024x1536"`

              - `"1536x1024"`

              - `"auto"`

        - `LocalShell object { type }`

          A tool that allows the model to execute shell commands in a local environment.

          - `type: "local_shell"`

            The type of the local shell tool. Always `local_shell`.

            - `"local_shell"`

        - `Shell object { type, allowed_callers, environment }`

          A tool that allows the model to execute shell commands.

          - `type: "shell"`

            The type of the shell tool. Always `shell`.

            - `"shell"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null`

            - `ContainerAuto object { type, file_ids, memory_limit, 2 more }`

              - `type: "container_auto"`

                Automatically creates a container for this request

                - `"container_auto"`

              - `file_ids: optional array of string`

                An optional list of uploaded files to make available to your code.

              - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                The memory limit for the container.

                - `"1g"`

                - `"4g"`

                - `"16g"`

                - `"64g"`

              - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                Network access policy for the container.

                - `ContainerNetworkPolicyDisabled object { type }`

                - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

              - `skills: optional array of SkillReference or InlineSkill`

                An optional list of skills referenced by id or inline data.

                - `SkillReference object { skill_id, type, version }`

                  - `skill_id: string`

                    The ID of the referenced skill.

                  - `type: "skill_reference"`

                    References a skill created with the /v1/skills endpoint.

                    - `"skill_reference"`

                  - `version: optional string`

                    Optional skill version. Use a positive integer or 'latest'. Omit for default.

                - `InlineSkill object { description, name, source, type }`

                  - `description: string`

                    The description of the skill.

                  - `name: string`

                    The name of the skill.

                  - `source: InlineSkillSource`

                    Inline skill payload

                    - `data: string`

                      Base64-encoded skill zip bundle.

                    - `media_type: "application/zip"`

                      The media type of the inline skill payload. Must be `application/zip`.

                      - `"application/zip"`

                    - `type: "base64"`

                      The type of the inline skill source. Must be `base64`.

                      - `"base64"`

                  - `type: "inline"`

                    Defines an inline skill for this request.

                    - `"inline"`

            - `LocalEnvironment object { type, skills }`

              - `type: "local"`

                Use a local computer environment.

                - `"local"`

              - `skills: optional array of LocalSkill`

                An optional list of skills.

                - `description: string`

                  The description of the skill.

                - `name: string`

                  The name of the skill.

                - `path: string`

                  The path to the directory containing the skill.

            - `ContainerReference object { container_id, type }`

              - `container_id: string`

                The ID of the referenced container.

              - `type: "container_reference"`

                References a container created with the /v1/containers endpoint

                - `"container_reference"`

        - `Custom object { name, type, allowed_callers, 4 more }`

          A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

          - `name: string`

            The name of the custom tool, used to identify it in tool calls.

          - `type: "custom"`

            The type of the custom tool. Always `custom`.

            - `"custom"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `async: optional boolean`

            Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

          - `defer_loading: optional boolean`

            Whether this tool should be deferred and discovered via tool search.

          - `description: optional string`

            Optional description of the custom tool, used to provide more context.

          - `format: optional CustomToolInputFormat`

            The input format for the custom tool. Default is unconstrained text.

            - `Text object { type }`

              Unconstrained free-form text.

              - `type: "text"`

                Unconstrained text format. Always `text`.

                - `"text"`

            - `Grammar object { definition, syntax, type }`

              A grammar defined by the user.

              - `definition: string`

                The grammar definition.

              - `syntax: "lark" or "regex"`

                The syntax of the grammar definition. One of `lark` or `regex`.

                - `"lark"`

                - `"regex"`

              - `type: "grammar"`

                Grammar format. Always `grammar`.

                - `"grammar"`

        - `Namespace object { description, name, tools, type }`

          Groups function/custom tools under a shared namespace.

          - `description: string`

            A description of the namespace shown to the model.

          - `name: string`

            The namespace name used in tool calls (for example, `crm`).

          - `tools: array of Function { name, type, allowed_callers, 6 more }  or Custom { name, type, allowed_callers, 4 more }`

            The function/custom tools available inside this namespace.

            - `Function object { name, type, allowed_callers, 6 more }`

              - `name: string`

              - `type: "function"`

                - `"function"`

              - `allowed_callers: optional array of "direct" or "programmatic" or null`

                The tool invocation context(s).

                - `"direct"`

                - `"programmatic"`

              - `async: optional boolean`

                Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

              - `defer_loading: optional boolean`

                Whether this function should be deferred and discovered via tool search.

              - `description: optional string or null`

              - `output_schema: optional map[unknown] or null`

                A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

              - `parameters: optional unknown or null`

              - `strict: optional boolean or null`

                Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

            - `Custom object { name, type, allowed_callers, 4 more }`

              A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

              - `name: string`

                The name of the custom tool, used to identify it in tool calls.

              - `type: "custom"`

                The type of the custom tool. Always `custom`.

                - `"custom"`

              - `allowed_callers: optional array of "direct" or "programmatic" or null`

                The tool invocation context(s).

                - `"direct"`

                - `"programmatic"`

              - `async: optional boolean`

                Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

              - `defer_loading: optional boolean`

                Whether this tool should be deferred and discovered via tool search.

              - `description: optional string`

                Optional description of the custom tool, used to provide more context.

              - `format: optional CustomToolInputFormat`

                The input format for the custom tool. Default is unconstrained text.

          - `type: "namespace"`

            The type of the tool. Always `namespace`.

            - `"namespace"`

        - `ToolSearch object { type, description, execution, parameters }`

          Hosted or BYOT tool search configuration for deferred tools.

          - `type: "tool_search"`

            The type of the tool. Always `tool_search`.

            - `"tool_search"`

          - `description: optional string or null`

            Description shown to the model for a client-executed tool search tool.

          - `execution: optional "server" or "client"`

            Whether tool search is executed by the server or by the client.

            - `"server"`

            - `"client"`

          - `parameters: optional unknown or null`

            Parameter schema for a client-executed tool search tool.

        - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }`

          This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search).

          - `type: "web_search_preview" or "web_search_preview_2025_03_11"`

            The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`.

            - `"web_search_preview"`

            - `"web_search_preview_2025_03_11"`

          - `search_content_types: optional array of "text" or "image"`

            - `"text"`

            - `"image"`

          - `search_context_size: optional "low" or "medium" or "high"`

            High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

            - `"low"`

            - `"medium"`

            - `"high"`

          - `user_location: optional object { type, city, country, 2 more }  or null`

            The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields.

            - `type: "approximate"`

              The type of location approximation. Always `approximate`.

              - `"approximate"`

            - `city: optional string or null`

              Free text input for the city of the user, e.g. `San Francisco`.

            - `country: optional string or null`

              The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

            - `region: optional string or null`

              Free text input for the region of the user, e.g. `California`.

            - `timezone: optional string or null`

              The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

        - `ApplyPatch object { type, allowed_callers }`

          Allows the assistant to create, delete, or update files using unified diffs.

          - `type: "apply_patch"`

            The type of the tool. Always `apply_patch`.

            - `"apply_patch"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

      - `top_p: optional number`

        An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

- `metadata: optional Metadata or null`

  Set of 16 key-value pairs that can be attached to an object. This can be
  useful for storing additional information about the object in a structured
  format, and querying for objects via API or the dashboard.

  Keys are strings with a maximum length of 64 characters. Values are strings
  with a maximum length of 512 characters.

- `name: optional string`

  The name of the run.

### Returns

- `id: string`

  Unique identifier for the evaluation run.

- `created_at: number`

  Unix timestamp (in seconds) when the evaluation run was created.

- `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }`

  Information about the run's data source.

  - `CreateEvalJSONLRunDataSource object { source, type }`

    A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

      Determines what populates the `item` namespace in the data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

    - `type: "jsonl"`

      The type of data source. Always `jsonl`.

      - `"jsonl"`

  - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

    A CompletionsRunDataSource object describing a model sampling configuration.

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

      Determines what populates the `item` namespace in this run's data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

      - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

        A StoredCompletionsRunDataSource configuration describing a set of filters

        - `type: "stored_completions"`

          The type of source. Always `stored_completions`.

          - `"stored_completions"`

        - `created_after: optional number or null`

          An optional Unix timestamp to filter items created after this time.

        - `created_before: optional number or null`

          An optional Unix timestamp to filter items created before this time.

        - `limit: optional number or null`

          An optional maximum number of items to return.

        - `metadata: optional Metadata or null`

          Set of 16 key-value pairs that can be attached to an object. This can be
          useful for storing additional information about the object in a structured
          format, and querying for objects via API or the dashboard.

          Keys are strings with a maximum length of 64 characters. Values are strings
          with a maximum length of 512 characters.

        - `model: optional string or null`

          An optional model to filter by (e.g., 'gpt-6-astra').

    - `type: "completions"`

      The type of run data source. Always `completions`.

      - `"completions"`

    - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

      Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

      - `TemplateInputMessages object { template, type }`

        - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

          A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

          - `EasyInputMessage object { content, role, phase, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputMessageContentList`

              Text, image, or audio input to the model, used to generate a response.
              Can also contain previous assistant responses.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputMessageContentList = array of ResponseInputContent`

                A list of one or many input items to the model, containing different content
                types.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                  - `text: string`

                    The text input to the model.

                  - `type: "input_text"`

                    The type of the input item. Always `input_text`.

                    - `"input_text"`

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

                - `ResponseInputImage object { detail, type, file_id, 2 more }`

                  An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                  - `detail: ImageDetail`

                    The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                    - `"low"`

                    - `"high"`

                    - `"auto"`

                    - `"original"`

                  - `type: "input_image"`

                    The type of the input item. Always `input_image`.

                    - `"input_image"`

                  - `file_id: optional string or null`

                    The ID of the file to be sent to the model.

                  - `image_url: optional string or null`

                    The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

                - `ResponseInputFile object { type, detail, file_data, 4 more }`

                  A file input to the model.

                  - `type: "input_file"`

                    The type of the input item. Always `input_file`.

                    - `"input_file"`

                  - `detail: optional "auto" or "low" or "high"`

                    The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                    - `"auto"`

                    - `"low"`

                    - `"high"`

                  - `file_data: optional string`

                    The content of the file to be sent to the model.

                  - `file_id: optional string or null`

                    The ID of the file to be sent to the model.

                  - `file_url: optional string`

                    The URL of the file to be sent to the model.

                  - `filename: optional string`

                    The name of the file to be sent to the model.

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `phase: optional "commentary" or "final_answer" or null`

              Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
              For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
              phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

              - `"commentary"`

              - `"final_answer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

          - `EvalMessageObject object { content, role, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

              Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

              - `OutputText object { text, type }`

                A text output from the model.

                - `text: string`

                  The text output from the model.

                - `type: "output_text"`

                  The type of the output text. Always `output_text`.

                  - `"output_text"`

              - `InputImage object { image_url, type, detail }`

                An image input block used within EvalItem content arrays.

                - `image_url: string`

                  The URL of the image input.

                - `type: "input_image"`

                  The type of the image input. Always `input_image`.

                  - `"input_image"`

                - `detail: optional string`

                  The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

              - `ResponseInputAudio object { input_audio, type }`

                An audio input to the model.

                - `input_audio: object { data, format }`

                  - `data: string`

                    Base64-encoded audio data.

                  - `format: "mp3" or "wav"`

                    The format of the audio data. Currently supported formats are `mp3` and
                    `wav`.

                    - `"mp3"`

                    - `"wav"`

                - `type: "input_audio"`

                  The type of the input item. Always `input_audio`.

                  - `"input_audio"`

              - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                A list of inputs, each of which may be either an input text, output text, input
                image, or input audio object.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

        - `type: "template"`

          The type of input messages. Always `template`.

          - `"template"`

      - `ItemReferenceInputMessages object { item_reference, type }`

        - `item_reference: string`

          A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

        - `type: "item_reference"`

          The type of input messages. Always `item_reference`.

          - `"item_reference"`

    - `model: optional string`

      The name of the model to use for generating completions (e.g. "o3-mini").

    - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

      - `max_completion_tokens: optional number`

        The maximum number of tokens in the generated output.

      - `reasoning_effort: optional ReasoningEffort or null`

        Constrains effort on reasoning for reasoning models. Currently supported
        values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
        Reducing reasoning effort can result in faster responses and fewer tokens
        used on reasoning in a response. Not all reasoning models support every
        value. See the
        [reasoning guide](/api/docs/guides/reasoning)
        for model-specific support.

        - `"none"`

        - `"minimal"`

        - `"low"`

        - `"medium"`

        - `"high"`

        - `"xhigh"`

        - `"max"`

      - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

        An object specifying the format that the model must output.

        Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
        Structured Outputs which ensures the model will match your supplied JSON
        schema. Learn more in the [Structured Outputs
        guide](/api/docs/guides/structured-outputs).

        Setting to `{ "type": "json_object" }` enables the older JSON mode, which
        ensures the message the model generates is valid JSON. Using `json_schema`
        is preferred for models that support it.

        - `ResponseFormatText object { type }`

          Default response format. Used to generate text responses.

          - `type: "text"`

            The type of response format being defined. Always `text`.

            - `"text"`

        - `ResponseFormatJSONSchema object { json_schema, type }`

          JSON Schema response format. Used to generate structured JSON responses.
          Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

          - `json_schema: object { name, description, schema, strict }`

            Structured Outputs configuration options, including a JSON Schema.

            - `name: string`

              The name of the response format. Must be a-z, A-Z, 0-9, or contain
              underscores and dashes, with a maximum length of 64.

            - `description: optional string`

              A description of what the response format is for, used by the model to
              determine how to respond in the format.

            - `schema: optional map[unknown]`

              The schema for the response format, described as a JSON Schema object.
              Learn how to build JSON schemas [here](https://json-schema.org/).

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the output.
              If set to true, the model will always follow the exact schema defined
              in the `schema` field. Only a subset of JSON Schema is supported when
              `strict` is `true`. To learn more, read the [Structured Outputs
              guide](/api/docs/guides/structured-outputs).

          - `type: "json_schema"`

            The type of response format being defined. Always `json_schema`.

            - `"json_schema"`

        - `ResponseFormatJSONObject object { type }`

          JSON object response format. An older method of generating JSON responses.
          Using `json_schema` is recommended for models that support it. Note that the
          model will not generate JSON without a system or user message instructing it
          to do so.

          - `type: "json_object"`

            The type of response format being defined. Always `json_object`.

            - `"json_object"`

      - `seed: optional number`

        A seed value to initialize the randomness, during sampling.

      - `temperature: optional number`

        A higher temperature increases randomness in the outputs.

      - `tools: optional array of ChatCompletionFunctionTool`

        A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

        - `function: FunctionDefinition`

          - `name: string`

            The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

          - `description: optional string`

            A description of what the function does, used by the model to choose when and how to call the function.

          - `parameters: optional FunctionParameters`

            The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

            Omitting `parameters` defines a function with an empty parameter list.

          - `strict: optional boolean or null`

            Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

        - `type: "function"`

          The type of the tool. Currently, only `function` is supported.

          - `"function"`

      - `top_p: optional number`

        An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

  - `ResponsesRunDataSource object { source, type, input_messages, 2 more }`

    A ResponsesRunDataSource object describing a model sampling configuration.

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or EvalResponsesSource { type, created_after, created_before, 8 more }`

      Determines what populates the `item` namespace in this run's data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

      - `EvalResponsesSource object { type, created_after, created_before, 8 more }`

        A EvalResponsesSource object describing a run data source configuration.

        - `type: "responses"`

          The type of run data source. Always `responses`.

          - `"responses"`

        - `created_after: optional number or null`

          Only include items created after this timestamp (inclusive). This is a query parameter used to select responses.

        - `created_before: optional number or null`

          Only include items created before this timestamp (inclusive). This is a query parameter used to select responses.

        - `instructions_search: optional string or null`

          Optional string to search the 'instructions' field. This is a query parameter used to select responses.

        - `metadata: optional unknown or null`

          Metadata filter for the responses. This is a query parameter used to select responses.

        - `model: optional string or null`

          The name of the model to find responses for. This is a query parameter used to select responses.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

        - `temperature: optional number or null`

          Sampling temperature. This is a query parameter used to select responses.

        - `tools: optional array of string or null`

          List of tool names. This is a query parameter used to select responses.

        - `top_p: optional number or null`

          Nucleus sampling parameter. This is a query parameter used to select responses.

        - `users: optional array of string or null`

          List of user identifiers. This is a query parameter used to select responses.

    - `type: "responses"`

      The type of run data source. Always `responses`.

      - `"responses"`

    - `input_messages: optional InputMessagesTemplate { template, type }  or InputMessagesItemReference { item_reference, type }`

      Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

      - `InputMessagesTemplate object { template, type }`

        - `template: array of ChatMessage { content, role }  or EvalMessageObject { content, role, type }`

          A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

          - `ChatMessage object { content, role }`

            - `content: string`

              The content of the message.

            - `role: string`

              The role of the message (e.g. "system", "assistant", "user").

          - `EvalMessageObject object { content, role, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

              Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

              - `OutputText object { text, type }`

                A text output from the model.

                - `text: string`

                  The text output from the model.

                - `type: "output_text"`

                  The type of the output text. Always `output_text`.

                  - `"output_text"`

              - `InputImage object { image_url, type, detail }`

                An image input block used within EvalItem content arrays.

                - `image_url: string`

                  The URL of the image input.

                - `type: "input_image"`

                  The type of the image input. Always `input_image`.

                  - `"input_image"`

                - `detail: optional string`

                  The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

              - `ResponseInputAudio object { input_audio, type }`

                An audio input to the model.

              - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                A list of inputs, each of which may be either an input text, output text, input
                image, or input audio object.

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

        - `type: "template"`

          The type of input messages. Always `template`.

          - `"template"`

      - `InputMessagesItemReference object { item_reference, type }`

        - `item_reference: string`

          A reference to a variable in the `item` namespace. Ie, "item.name"

        - `type: "item_reference"`

          The type of input messages. Always `item_reference`.

          - `"item_reference"`

    - `model: optional string`

      The name of the model to use for generating completions (e.g. "o3-mini").

    - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }`

      - `max_completion_tokens: optional number`

        The maximum number of tokens in the generated output.

      - `reasoning_effort: optional ReasoningEffort or null`

        Constrains effort on reasoning for reasoning models. Currently supported
        values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
        Reducing reasoning effort can result in faster responses and fewer tokens
        used on reasoning in a response. Not all reasoning models support every
        value. See the
        [reasoning guide](/api/docs/guides/reasoning)
        for model-specific support.

      - `seed: optional number`

        A seed value to initialize the randomness, during sampling.

      - `temperature: optional number`

        A higher temperature increases randomness in the outputs.

      - `text: optional object { format }`

        Configuration options for a text response from the model. Can be plain
        text or structured JSON data. Learn more:

        - [Text inputs and outputs](/api/docs/guides/text)
        - [Structured Outputs](/api/docs/guides/structured-outputs)

        - `format: optional ResponseFormatTextConfig`

          An object specifying the format that the model must output.

          Configuring `{ "type": "json_schema" }` enables Structured Outputs,
          which ensures the model will match your supplied JSON schema. Learn more in the
          [Structured Outputs guide](/api/docs/guides/structured-outputs).

          The default format is `{ "type": "text" }` with no additional options.

          **Not recommended for gpt-4o and newer models:**

          Setting to `{ "type": "json_object" }` enables the older JSON mode, which
          ensures the message the model generates is valid JSON. Using `json_schema`
          is preferred for models that support it.

          - `ResponseFormatText object { type }`

            Default response format. Used to generate text responses.

          - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }`

            JSON Schema response format. Used to generate structured JSON responses.
            Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

            - `name: string`

              The name of the response format. Must be a-z, A-Z, 0-9, or contain
              underscores and dashes, with a maximum length of 64.

            - `schema: map[unknown]`

              The schema for the response format, described as a JSON Schema object.
              Learn how to build JSON schemas [here](https://json-schema.org/).

            - `type: "json_schema"`

              The type of response format being defined. Always `json_schema`.

              - `"json_schema"`

            - `description: optional string`

              A description of what the response format is for, used by the model to
              determine how to respond in the format.

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the output.
              If set to true, the model will always follow the exact schema defined
              in the `schema` field. Only a subset of JSON Schema is supported when
              `strict` is `true`. To learn more, read the [Structured Outputs
              guide](/api/docs/guides/structured-outputs).

          - `ResponseFormatJSONObject object { type }`

            JSON object response format. An older method of generating JSON responses.
            Using `json_schema` is recommended for models that support it. Note that the
            model will not generate JSON without a system or user message instructing it
            to do so.

      - `tools: optional array of Function { name, parameters, strict, 6 more }  or FileSearch { type, vector_store_ids, filters, 2 more }  or Computer { type }  or 13 more`

        An array of tools the model may call while generating a response. You
        can specify which tool to use by setting the `tool_choice` parameter.

        The two categories of tools you can provide the model are:

        - **Built-in tools**: Tools that are provided by OpenAI that extend the
          model's capabilities, like [web search](/api/docs/guides/tools-web-search)
          or [file search](/api/docs/guides/tools-file-search). Learn more about
          [built-in tools](/api/docs/guides/tools).
        - **Function calls (custom tools)**: Functions that are defined by you,
          enabling the model to call your own code. Learn more about
          [function calling](/api/docs/guides/function-calling).

        - `Function object { name, parameters, strict, 6 more }`

          Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling).

          - `name: string`

            The name of the function to call.

          - `parameters: map[unknown] or null`

            A JSON schema object describing the parameters of the function.

          - `strict: boolean or null`

            Whether strict parameter validation is enforced for this function tool.

          - `type: "function"`

            The type of the function tool. Always `function`.

            - `"function"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `async: optional boolean`

          - `defer_loading: optional boolean`

            Whether this function is deferred and loaded via tool search.

          - `description: optional string or null`

            A description of the function. Used by the model to determine whether or not to call the function.

          - `output_schema: optional map[unknown] or null`

            A JSON schema object describing the JSON value encoded in string outputs for this function.

        - `FileSearch object { type, vector_store_ids, filters, 2 more }`

          A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search).

          - `type: "file_search"`

            The type of the file search tool. Always `file_search`.

            - `"file_search"`

          - `vector_store_ids: array of string`

            The IDs of the vector stores to search.

          - `filters: optional ComparisonFilter or CompoundFilter or null`

            A filter to apply.

            - `ComparisonFilter object { key, type, value }`

              A filter used to compare a specified attribute key to a given value using a defined comparison operation.

              - `key: string`

                The key to compare against the value.

              - `type: "eq" or "ne" or "gt" or 5 more`

                Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`.

                - `eq`: equals
                - `ne`: not equal
                - `gt`: greater than
                - `gte`: greater than or equal
                - `lt`: less than
                - `lte`: less than or equal
                - `in`: in
                - `nin`: not in

                - `"eq"`

                - `"ne"`

                - `"gt"`

                - `"gte"`

                - `"lt"`

                - `"lte"`

                - `"in"`

                - `"nin"`

              - `value: string or number or boolean or array of string or number`

                The value to compare against the attribute key; supports string, number, or boolean types.

                - `string`

                - `number`

                - `boolean`

                - `array of string or number`

                  - `string`

                  - `number`

            - `CompoundFilter object { filters, type }`

              Combine multiple filters using `and` or `or`.

              - `filters: array of ComparisonFilter or CompoundFilter`

                Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.

                - `ComparisonFilter object { key, type, value }`

                  A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                - `CompoundFilter object { filters, type }`

                  Combine multiple filters using `and` or `or`.

              - `type: "and" or "or"`

                Type of operation: `and` or `or`.

                - `"and"`

                - `"or"`

          - `max_num_results: optional number`

            The maximum number of results to return. This number should be between 1 and 50 inclusive.

          - `ranking_options: optional object { hybrid_search, ranker, score_threshold }`

            Ranking options for search.

            - `hybrid_search: optional object { embedding_weight, text_weight }`

              Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

              - `embedding_weight: number`

                The weight of the embedding in the reciprocal ranking fusion.

              - `text_weight: number`

                The weight of the text in the reciprocal ranking fusion.

            - `ranker: optional "auto" or "default-2024-11-15"`

              The ranker to use for the file search.

              - `"auto"`

              - `"default-2024-11-15"`

            - `score_threshold: optional number`

              The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

        - `Computer object { type }`

          A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

          - `type: "computer"`

            The type of the computer tool. Always `computer`.

            - `"computer"`

        - `ComputerUsePreview object { display_height, display_width, environment, type }`

          A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

          - `display_height: number`

            The height of the computer display.

          - `display_width: number`

            The width of the computer display.

          - `environment: "windows" or "mac" or "linux" or 2 more`

            The type of computer environment to control.

            - `"windows"`

            - `"mac"`

            - `"linux"`

            - `"ubuntu"`

            - `"browser"`

          - `type: "computer_use_preview"`

            The type of the computer use tool. Always `computer_use_preview`.

            - `"computer_use_preview"`

        - `WebSearch object { type, external_web_access, filters, 2 more }`

          Search the Internet for sources related to the prompt. Learn more about the
          [web search tool](/api/docs/guides/tools-web-search).

          - `type: "web_search" or "web_search_2025_08_26"`

            The type of the web search tool. One of `web_search` or `web_search_2025_08_26`.

            - `"web_search"`

            - `"web_search_2025_08_26"`

          - `external_web_access: optional boolean`

            Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

          - `filters: optional object { allowed_domains }  or null`

            Filters for the search.

            - `allowed_domains: optional array of string or null`

              Allowed domains for the search. If not provided, all domains are allowed.
              Subdomains of the provided domains are allowed as well.

              Example: `["pubmed.ncbi.nlm.nih.gov"]`

          - `search_context_size: optional "low" or "medium" or "high"`

            High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

            - `"low"`

            - `"medium"`

            - `"high"`

          - `user_location: optional object { city, country, region, 2 more }  or null`

            The approximate location of the user. If omitted or null, defaults to the
            United States. To avoid this fallback, pass `{"type": "approximate"}` without
            location fields. To localize results, provide the relevant location fields.

            - `city: optional string or null`

              Free text input for the city of the user, e.g. `San Francisco`.

            - `country: optional string or null`

              The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

            - `region: optional string or null`

              Free text input for the region of the user, e.g. `California`.

            - `timezone: optional string or null`

              The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

            - `type: optional "approximate"`

              The type of location approximation. Always `approximate`.

              - `"approximate"`

        - `Mcp object { server_label, type, allowed_callers, 9 more }`

          Give the model access to additional tools via remote Model Context Protocol
          (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp).

          - `server_label: string`

            A label for this MCP server, used to identify it in tool calls.

          - `type: "mcp"`

            The type of the MCP tool. Always `mcp`.

            - `"mcp"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names }  or null`

            List of allowed tool names or a filter object.

            - `McpAllowedTools = array of string`

              A string array of allowed tool names

            - `McpToolFilter object { read_only, tool_names }`

              A filter object to specify which tools are allowed.

              - `read_only: optional boolean`

                Indicates whether or not a tool modifies data or is read-only. If an
                MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                it will match this filter.

              - `tool_names: optional array of string`

                List of allowed tool names.

          - `authorization: optional string`

            An OAuth access token that can be used with a remote MCP server, either
            with a custom MCP server URL or a service connector. Your application
            must handle the OAuth authorization flow and provide the token here.

          - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more`

            Identifier for service connectors, like those available in ChatGPT. One of
            `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more
            about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors).

            This field is deprecated for models released after September 1, 2026.
            Use `server_url` to connect to a remote MCP server, or `tunnel_id` to
            connect through a Secure MCP Tunnel.

            Currently supported `connector_id` values are:

            - Dropbox: `connector_dropbox`
            - Gmail: `connector_gmail`
            - Google Calendar: `connector_googlecalendar`
            - Google Drive: `connector_googledrive`
            - Microsoft Teams: `connector_microsoftteams`
            - Outlook Calendar: `connector_outlookcalendar`
            - Outlook Email: `connector_outlookemail`
            - SharePoint: `connector_sharepoint`

            - `"connector_dropbox"`

            - `"connector_gmail"`

            - `"connector_googlecalendar"`

            - `"connector_googledrive"`

            - `"connector_microsoftteams"`

            - `"connector_outlookcalendar"`

            - `"connector_outlookemail"`

            - `"connector_sharepoint"`

          - `defer_loading: optional boolean`

            Whether this MCP tool is deferred and discovered via tool search.

          - `headers: optional map[string] or null`

            Optional HTTP headers to send to the MCP server. Use for authentication
            or other purposes.

          - `require_approval: optional McpToolApprovalFilter { always, never }  or "always" or "never" or null`

            Specify which of the MCP server's tools require approval.

            - `McpToolApprovalFilter object { always, never }`

              Specify which of the MCP server's tools require approval. Can be
              `always`, `never`, or a filter object associated with tools
              that require approval.

              - `always: optional object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

              - `never: optional object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

            - `McpToolApprovalSetting = "always" or "never"`

              Specify a single approval policy for all tools. One of `always` or
              `never`. When set to `always`, all tools will require approval. When
              set to `never`, all tools will not require approval.

              - `"always"`

              - `"never"`

          - `server_description: optional string`

            Optional description of the MCP server, used to provide more context.

          - `server_url: optional string`

            The URL for the MCP server. One of `server_url`, `connector_id`, or
            `tunnel_id` must be provided.

          - `tunnel_id: optional string`

            The Secure MCP Tunnel ID to use instead of a direct server URL. One of
            `server_url`, `connector_id`, or `tunnel_id` must be provided.

        - `CodeInterpreter object { container, type, allowed_callers }`

          A tool that runs Python code to help generate a response to a prompt.

          - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }`

            The code interpreter container. Can be a container ID or an object that
            specifies uploaded file IDs to make available to your code, along with an
            optional `memory_limit` setting.

            - `string`

              The container ID.

            - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }`

              Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

              - `type: "auto"`

                Always `auto`.

                - `"auto"`

              - `file_ids: optional array of string`

                An optional list of uploaded files to make available to your code.

              - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                The memory limit for the code interpreter container.

                - `"1g"`

                - `"4g"`

                - `"16g"`

                - `"64g"`

              - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                Network access policy for the container.

                - `ContainerNetworkPolicyDisabled object { type }`

                  - `type: "disabled"`

                    Disable outbound network access. Always `disabled`.

                    - `"disabled"`

                - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                  - `allowed_domains: array of string`

                    A list of allowed domains when type is `allowlist`.

                  - `type: "allowlist"`

                    Allow outbound network access only to specified domains. Always `allowlist`.

                    - `"allowlist"`

                  - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret`

                    Optional domain-scoped secrets for allowlisted domains.

                    - `domain: string`

                      The domain associated with the secret.

                    - `name: string`

                      The name of the secret to inject for the domain.

                    - `value: string`

                      The secret value to inject for the domain.

          - `type: "code_interpreter"`

            The type of the code interpreter tool. Always `code_interpreter`.

            - `"code_interpreter"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

        - `ProgrammaticToolCalling object { type }`

          - `type: "programmatic_tool_calling"`

            The type of the tool. Always `programmatic_tool_calling`.

            - `"programmatic_tool_calling"`

        - `ImageGeneration object { type, action, background, 9 more }`

          A tool that generates images using the GPT image models.

          - `type: "image_generation"`

            The type of the image generation tool. Always `image_generation`.

            - `"image_generation"`

          - `action: optional "generate" or "edit" or "auto"`

            Whether to generate a new image or edit an existing image. Default: `auto`.

            - `"generate"`

            - `"edit"`

            - `"auto"`

          - `background: optional "transparent" or "opaque" or "auto"`

            Set the background of the generated image. One of `transparent`, `opaque`,
            or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including
            their `2026-09-08` snapshots, support `opaque` and `transparent`
            backgrounds. Transparent backgrounds are available for supported GPT Image
            models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in
            preview. When using `transparent`, set the output format to `png` or `webp`.
            Default: `auto`.

            - `"transparent"`

            - `"opaque"`

            - `"auto"`

          - `input_fidelity: optional "high" or "low" or null`

            Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models.

            - `"high"`

            - `"low"`

          - `input_image_mask: optional object { file_id, image_url }`

            Optional mask for inpainting. Contains `image_url`
            (string, optional) and `file_id` (string, optional).

            - `file_id: optional string`

              File ID for the mask image.

            - `image_url: optional string`

              Base64-encoded mask image.

          - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

            The image generation model to use. One of `gpt-image-1`,
            `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
            `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
            `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
            `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
            `gpt-image-1`.

            - `string`

            - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

              The image generation model to use. One of `gpt-image-1`,
              `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
              `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
              `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
              `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
              `gpt-image-1`.

              - `"gpt-image-1"`

              - `"gpt-image-1-mini"`

              - `"gpt-image-1.5"`

              - `"gpt-image-2"`

              - `"gpt-image-2-2026-04-21"`

              - `"gpt-image-2.5-sunburst"`

              - `"gpt-image-2.5-sunburst-2026-09-08"`

              - `"gpt-image-2.5-flare"`

              - `"gpt-image-2.5-flare-2026-09-08"`

          - `moderation: optional "auto" or "low"`

            Moderation level for the generated image. Default: `auto`.

            - `"auto"`

            - `"low"`

          - `output_compression: optional number`

            Compression level for the output image. Default: 100.

          - `output_format: optional "png" or "webp" or "jpeg"`

            The output format of the generated image. One of `png`, `webp`, or
            `jpeg`. Default: `png`.

            - `"png"`

            - `"webp"`

            - `"jpeg"`

          - `partial_images: optional number`

            Number of partial images to generate in streaming mode, from 0 (default value) to 3.

          - `quality: optional "low" or "medium" or "high" or 3 more`

            The quality of the generated image. The GPT image models support `low`,
            `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`,
            including their `2026-09-08` snapshots, also support `xhigh` and `max`.
            Default: `auto`.

            - `"low"`

            - `"medium"`

            - `"high"`

            - `"xhigh"`

            - `"max"`

            - `"auto"`

          - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"`

            The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

            - `string`

            - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"`

              The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

              - `"1024x1024"`

              - `"1024x1536"`

              - `"1536x1024"`

              - `"auto"`

        - `LocalShell object { type }`

          A tool that allows the model to execute shell commands in a local environment.

          - `type: "local_shell"`

            The type of the local shell tool. Always `local_shell`.

            - `"local_shell"`

        - `Shell object { type, allowed_callers, environment }`

          A tool that allows the model to execute shell commands.

          - `type: "shell"`

            The type of the shell tool. Always `shell`.

            - `"shell"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null`

            - `ContainerAuto object { type, file_ids, memory_limit, 2 more }`

              - `type: "container_auto"`

                Automatically creates a container for this request

                - `"container_auto"`

              - `file_ids: optional array of string`

                An optional list of uploaded files to make available to your code.

              - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                The memory limit for the container.

                - `"1g"`

                - `"4g"`

                - `"16g"`

                - `"64g"`

              - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                Network access policy for the container.

                - `ContainerNetworkPolicyDisabled object { type }`

                - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

              - `skills: optional array of SkillReference or InlineSkill`

                An optional list of skills referenced by id or inline data.

                - `SkillReference object { skill_id, type, version }`

                  - `skill_id: string`

                    The ID of the referenced skill.

                  - `type: "skill_reference"`

                    References a skill created with the /v1/skills endpoint.

                    - `"skill_reference"`

                  - `version: optional string`

                    Optional skill version. Use a positive integer or 'latest'. Omit for default.

                - `InlineSkill object { description, name, source, type }`

                  - `description: string`

                    The description of the skill.

                  - `name: string`

                    The name of the skill.

                  - `source: InlineSkillSource`

                    Inline skill payload

                    - `data: string`

                      Base64-encoded skill zip bundle.

                    - `media_type: "application/zip"`

                      The media type of the inline skill payload. Must be `application/zip`.

                      - `"application/zip"`

                    - `type: "base64"`

                      The type of the inline skill source. Must be `base64`.

                      - `"base64"`

                  - `type: "inline"`

                    Defines an inline skill for this request.

                    - `"inline"`

            - `LocalEnvironment object { type, skills }`

              - `type: "local"`

                Use a local computer environment.

                - `"local"`

              - `skills: optional array of LocalSkill`

                An optional list of skills.

                - `description: string`

                  The description of the skill.

                - `name: string`

                  The name of the skill.

                - `path: string`

                  The path to the directory containing the skill.

            - `ContainerReference object { container_id, type }`

              - `container_id: string`

                The ID of the referenced container.

              - `type: "container_reference"`

                References a container created with the /v1/containers endpoint

                - `"container_reference"`

        - `Custom object { name, type, allowed_callers, 4 more }`

          A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

          - `name: string`

            The name of the custom tool, used to identify it in tool calls.

          - `type: "custom"`

            The type of the custom tool. Always `custom`.

            - `"custom"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `async: optional boolean`

            Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

          - `defer_loading: optional boolean`

            Whether this tool should be deferred and discovered via tool search.

          - `description: optional string`

            Optional description of the custom tool, used to provide more context.

          - `format: optional CustomToolInputFormat`

            The input format for the custom tool. Default is unconstrained text.

            - `Text object { type }`

              Unconstrained free-form text.

              - `type: "text"`

                Unconstrained text format. Always `text`.

                - `"text"`

            - `Grammar object { definition, syntax, type }`

              A grammar defined by the user.

              - `definition: string`

                The grammar definition.

              - `syntax: "lark" or "regex"`

                The syntax of the grammar definition. One of `lark` or `regex`.

                - `"lark"`

                - `"regex"`

              - `type: "grammar"`

                Grammar format. Always `grammar`.

                - `"grammar"`

        - `Namespace object { description, name, tools, type }`

          Groups function/custom tools under a shared namespace.

          - `description: string`

            A description of the namespace shown to the model.

          - `name: string`

            The namespace name used in tool calls (for example, `crm`).

          - `tools: array of Function { name, type, allowed_callers, 6 more }  or Custom { name, type, allowed_callers, 4 more }`

            The function/custom tools available inside this namespace.

            - `Function object { name, type, allowed_callers, 6 more }`

              - `name: string`

              - `type: "function"`

                - `"function"`

              - `allowed_callers: optional array of "direct" or "programmatic" or null`

                The tool invocation context(s).

                - `"direct"`

                - `"programmatic"`

              - `async: optional boolean`

                Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

              - `defer_loading: optional boolean`

                Whether this function should be deferred and discovered via tool search.

              - `description: optional string or null`

              - `output_schema: optional map[unknown] or null`

                A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

              - `parameters: optional unknown or null`

              - `strict: optional boolean or null`

                Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

            - `Custom object { name, type, allowed_callers, 4 more }`

              A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

              - `name: string`

                The name of the custom tool, used to identify it in tool calls.

              - `type: "custom"`

                The type of the custom tool. Always `custom`.

                - `"custom"`

              - `allowed_callers: optional array of "direct" or "programmatic" or null`

                The tool invocation context(s).

                - `"direct"`

                - `"programmatic"`

              - `async: optional boolean`

                Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

              - `defer_loading: optional boolean`

                Whether this tool should be deferred and discovered via tool search.

              - `description: optional string`

                Optional description of the custom tool, used to provide more context.

              - `format: optional CustomToolInputFormat`

                The input format for the custom tool. Default is unconstrained text.

          - `type: "namespace"`

            The type of the tool. Always `namespace`.

            - `"namespace"`

        - `ToolSearch object { type, description, execution, parameters }`

          Hosted or BYOT tool search configuration for deferred tools.

          - `type: "tool_search"`

            The type of the tool. Always `tool_search`.

            - `"tool_search"`

          - `description: optional string or null`

            Description shown to the model for a client-executed tool search tool.

          - `execution: optional "server" or "client"`

            Whether tool search is executed by the server or by the client.

            - `"server"`

            - `"client"`

          - `parameters: optional unknown or null`

            Parameter schema for a client-executed tool search tool.

        - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }`

          This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search).

          - `type: "web_search_preview" or "web_search_preview_2025_03_11"`

            The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`.

            - `"web_search_preview"`

            - `"web_search_preview_2025_03_11"`

          - `search_content_types: optional array of "text" or "image"`

            - `"text"`

            - `"image"`

          - `search_context_size: optional "low" or "medium" or "high"`

            High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

            - `"low"`

            - `"medium"`

            - `"high"`

          - `user_location: optional object { type, city, country, 2 more }  or null`

            The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields.

            - `type: "approximate"`

              The type of location approximation. Always `approximate`.

              - `"approximate"`

            - `city: optional string or null`

              Free text input for the city of the user, e.g. `San Francisco`.

            - `country: optional string or null`

              The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

            - `region: optional string or null`

              Free text input for the region of the user, e.g. `California`.

            - `timezone: optional string or null`

              The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

        - `ApplyPatch object { type, allowed_callers }`

          Allows the assistant to create, delete, or update files using unified diffs.

          - `type: "apply_patch"`

            The type of the tool. Always `apply_patch`.

            - `"apply_patch"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

      - `top_p: optional number`

        An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

- `error: EvalAPIError or null`

  An object representing an error response from the Eval API.

  - `code: string`

    The error code.

  - `message: string`

    The error message.

- `eval_id: string`

  The identifier of the associated evaluation.

- `metadata: Metadata or null`

  Set of 16 key-value pairs that can be attached to an object. This can be
  useful for storing additional information about the object in a structured
  format, and querying for objects via API or the dashboard.

  Keys are strings with a maximum length of 64 characters. Values are strings
  with a maximum length of 512 characters.

- `model: string or null`

  The model that is evaluated, if applicable.

- `name: string or null`

  The name of the evaluation run.

- `object: "eval.run"`

  The type of the object. Always "eval.run".

  - `"eval.run"`

- `per_model_usage: array of object { cached_tokens, completion_tokens, invocation_count, 3 more }  or null`

  Usage statistics for each model during the evaluation run.

  - `cached_tokens: number`

    The number of tokens retrieved from cache.

  - `completion_tokens: number`

    The number of completion tokens generated.

  - `invocation_count: number`

    The number of invocations.

  - `model_name: string`

    The name of the model.

  - `prompt_tokens: number`

    The number of prompt tokens used.

  - `total_tokens: number`

    The total number of tokens used.

- `per_testing_criteria_results: array of object { failed, passed, testing_criteria }  or null`

  Results per testing criteria applied during the evaluation run.

  - `failed: number`

    Number of tests failed for this criteria.

  - `passed: number`

    Number of tests passed for this criteria.

  - `testing_criteria: string`

    A description of the testing criteria.

- `report_url: string`

  The URL to the rendered evaluation run report on the UI dashboard.

- `result_counts: object { errored, failed, passed, total }`

  Counters summarizing the outcomes of the evaluation run.

  - `errored: number`

    Number of output items that resulted in an error.

  - `failed: number`

    Number of output items that failed to pass the evaluation.

  - `passed: number`

    Number of output items that passed the evaluation.

  - `total: number`

    Total number of executed output items.

- `status: string`

  The status of the evaluation run.

### Example

```http
curl https://api.openai.com/v1/evals/$EVAL_ID/runs \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $OPENAI_API_KEY" \
    -d '{
          "data_source": {
            "source": {
              "content": [
                {
                  "item": {
                    "foo": "bar"
                  }
                }
              ],
              "type": "file_content"
            },
            "type": "jsonl"
          }
        }'
```

#### Response

```json
{
  "id": "id",
  "created_at": 0,
  "data_source": {
    "source": {
      "content": [
        {
          "item": {
            "foo": "bar"
          },
          "sample": {
            "foo": "bar"
          }
        }
      ],
      "type": "file_content"
    },
    "type": "jsonl"
  },
  "error": {
    "code": "code",
    "message": "message"
  },
  "eval_id": "eval_id",
  "metadata": {
    "foo": "string"
  },
  "model": "model",
  "name": "name",
  "object": "eval.run",
  "per_model_usage": [
    {
      "cached_tokens": 0,
      "completion_tokens": 0,
      "invocation_count": 0,
      "model_name": "model_name",
      "prompt_tokens": 0,
      "total_tokens": 0
    }
  ],
  "per_testing_criteria_results": [
    {
      "failed": 0,
      "passed": 0,
      "testing_criteria": "testing_criteria"
    }
  ],
  "report_url": "https://example.com",
  "result_counts": {
    "errored": 0,
    "failed": 0,
    "passed": 0,
    "total": 0
  },
  "status": "status"
}
```

### Example

```http
curl https://api.openai.com/v1/evals/eval_67e579652b548190aaa83ada4b125f47/runs \
  -X POST \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"gpt-6-astra","data_source":{"type":"completions","input_messages":{"type":"template","template":[{"role":"developer","content":"Categorize a given news headline into one of the following topics: Technology, Markets, World, Business, or Sports.\n\n# Steps\n\n1. Analyze the content of the news headline to understand its primary focus.\n2. Extract the subject matter, identifying any key indicators or keywords.\n3. Use the identified indicators to determine the most suitable category out of the five options: Technology, Markets, World, Business, or Sports.\n4. Ensure only one category is selected per headline.\n\n# Output Format\n\nRespond with the chosen category as a single word. For instance: \"Technology\", \"Markets\", \"World\", \"Business\", or \"Sports\".\n\n# Examples\n\n**Input**: \"Apple Unveils New iPhone Model, Featuring Advanced AI Features\"  \n**Output**: \"Technology\"\n\n**Input**: \"Global Stocks Mixed as Investors Await Central Bank Decisions\"  \n**Output**: \"Markets\"\n\n**Input**: \"War in Ukraine: Latest Updates on Negotiation Status\"  \n**Output**: \"World\"\n\n**Input**: \"Microsoft in Talks to Acquire Gaming Company for $2 Billion\"  \n**Output**: \"Business\"\n\n**Input**: \"Manchester United Secures Win in Premier League Football Match\"  \n**Output**: \"Sports\" \n\n# Notes\n\n- If the headline appears to fit into more than one category, choose the most dominant theme.\n- Keywords or phrases such as \"stocks\", \"company acquisition\", \"match\", or technological brands can be good indicators for classification.\n"} , {"role":"user","content":"{{item.input}}"}]} ,"sampling_params":{"max_completions_tokens":2048},"model":"gpt-6-astra","source":{"type":"file_content","content":[{"item":{"input":"Tech Company Launches Advanced Artificial Intelligence Platform","ground_truth":"Technology"}}]}}}'
```

#### Response

```json
{
  "object": "eval.run",
  "id": "evalrun_67e57965b480819094274e3a32235e4c",
  "eval_id": "eval_67e579652b548190aaa83ada4b125f47",
  "report_url": "https://platform.openai.com/evaluations/eval_67e579652b548190aaa83ada4b125f47&run_id=evalrun_67e57965b480819094274e3a32235e4c",
  "status": "queued",
  "model": "gpt-6-astra",
  "name": "gpt-6-astra",
  "created_at": 1743092069,
  "result_counts": {
    "total": 0,
    "errored": 0,
    "failed": 0,
    "passed": 0
  },
  "per_model_usage": null,
  "per_testing_criteria_results": null,
  "data_source": {
    "type": "completions",
    "source": {
      "type": "file_content",
      "content": [
        {
          "item": {
            "input": "Tech Company Launches Advanced Artificial Intelligence Platform",
            "ground_truth": "Technology"
          }
        }
      ]
    },
    "input_messages": {
      "type": "template",
      "template": [
        {
          "type": "message",
          "role": "developer",
          "content": {
            "type": "input_text",
            "text": "Categorize a given news headline into one of the following topics: Technology, Markets, World, Business, or Sports.\n\n# Steps\n\n1. Analyze the content of the news headline to understand its primary focus.\n2. Extract the subject matter, identifying any key indicators or keywords.\n3. Use the identified indicators to determine the most suitable category out of the five options: Technology, Markets, World, Business, or Sports.\n4. Ensure only one category is selected per headline.\n\n# Output Format\n\nRespond with the chosen category as a single word. For instance: \"Technology\", \"Markets\", \"World\", \"Business\", or \"Sports\".\n\n# Examples\n\n**Input**: \"Apple Unveils New iPhone Model, Featuring Advanced AI Features\"  \n**Output**: \"Technology\"\n\n**Input**: \"Global Stocks Mixed as Investors Await Central Bank Decisions\"  \n**Output**: \"Markets\"\n\n**Input**: \"War in Ukraine: Latest Updates on Negotiation Status\"  \n**Output**: \"World\"\n\n**Input**: \"Microsoft in Talks to Acquire Gaming Company for $2 Billion\"  \n**Output**: \"Business\"\n\n**Input**: \"Manchester United Secures Win in Premier League Football Match\"  \n**Output**: \"Sports\" \n\n# Notes\n\n- If the headline appears to fit into more than one category, choose the most dominant theme.\n- Keywords or phrases such as \"stocks\", \"company acquisition\", \"match\", or technological brands can be good indicators for classification.\n"
          }
        },
        {
          "type": "message",
          "role": "user",
          "content": {
            "type": "input_text",
            "text": "{{item.input}}"
          }
        }
      ]
    },
    "model": "gpt-6-astra",
    "sampling_params": {
      "max_completions_tokens": 2048
    }
  },
  "error": null,
  "metadata": {}
}
```

## Delete eval run

**delete** `/evals/{eval_id}/runs/{run_id}`

Delete an eval run.

### Path Parameters

- `eval_id: string`

- `run_id: string`

### Returns

- `deleted: optional boolean`

- `object: optional string`

- `run_id: optional string`

### Example

```http
curl https://api.openai.com/v1/evals/$EVAL_ID/runs/$RUN_ID \
    -X DELETE \
    -H "Authorization: Bearer $OPENAI_API_KEY"
```

#### Response

```json
{
  "deleted": true,
  "object": "eval.run.deleted",
  "run_id": "evalrun_677469f564d48190807532a852da3afb"
}
```

### Example

```http
curl https://api.openai.com/v1/evals/eval_123abc/runs/evalrun_abc456 \
  -X DELETE \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json"
```

#### Response

```json
{
  "object": "eval.run.deleted",
  "deleted": true,
  "run_id": "evalrun_abc456"
}
```

## Get eval runs

**get** `/evals/{eval_id}/runs`

Get a list of runs for an evaluation.

### Path Parameters

- `eval_id: string`

### Query Parameters

- `after: optional string`

  Identifier for the last run from the previous pagination request.

- `limit: optional number`

  Number of runs to retrieve.

- `order: optional "asc" or "desc"`

  Sort order for runs by timestamp. Use `asc` for ascending order or `desc` for descending order. Defaults to `asc`.

  - `"asc"`

  - `"desc"`

- `status: optional "queued" or "in_progress" or "completed" or 2 more`

  Filter runs by status. One of `queued` | `in_progress` | `failed` | `completed` | `canceled`.

  - `"queued"`

  - `"in_progress"`

  - `"completed"`

  - `"canceled"`

  - `"failed"`

### Returns

- `data: array of object { id, created_at, data_source, 11 more }`

  An array of eval run objects.

  - `id: string`

    Unique identifier for the evaluation run.

  - `created_at: number`

    Unix timestamp (in seconds) when the evaluation run was created.

  - `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }`

    Information about the run's data source.

    - `CreateEvalJSONLRunDataSource object { source, type }`

      A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

        Determines what populates the `item` namespace in the data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

      - `type: "jsonl"`

        The type of data source. Always `jsonl`.

        - `"jsonl"`

    - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

      A CompletionsRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

          A StoredCompletionsRunDataSource configuration describing a set of filters

          - `type: "stored_completions"`

            The type of source. Always `stored_completions`.

            - `"stored_completions"`

          - `created_after: optional number or null`

            An optional Unix timestamp to filter items created after this time.

          - `created_before: optional number or null`

            An optional Unix timestamp to filter items created before this time.

          - `limit: optional number or null`

            An optional maximum number of items to return.

          - `metadata: optional Metadata or null`

            Set of 16 key-value pairs that can be attached to an object. This can be
            useful for storing additional information about the object in a structured
            format, and querying for objects via API or the dashboard.

            Keys are strings with a maximum length of 64 characters. Values are strings
            with a maximum length of 512 characters.

          - `model: optional string or null`

            An optional model to filter by (e.g., 'gpt-6-astra').

      - `type: "completions"`

        The type of run data source. Always `completions`.

        - `"completions"`

      - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `TemplateInputMessages object { template, type }`

          - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `EasyInputMessage object { content, role, phase, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputMessageContentList`

                Text, image, or audio input to the model, used to generate a response.
                Can also contain previous assistant responses.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputMessageContentList = array of ResponseInputContent`

                  A list of one or many input items to the model, containing different content
                  types.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                    - `text: string`

                      The text input to the model.

                    - `type: "input_text"`

                      The type of the input item. Always `input_text`.

                      - `"input_text"`

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputImage object { detail, type, file_id, 2 more }`

                    An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                    - `detail: ImageDetail`

                      The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                      - `"low"`

                      - `"high"`

                      - `"auto"`

                      - `"original"`

                    - `type: "input_image"`

                      The type of the input item. Always `input_image`.

                      - `"input_image"`

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `image_url: optional string or null`

                      The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputFile object { type, detail, file_data, 4 more }`

                    A file input to the model.

                    - `type: "input_file"`

                      The type of the input item. Always `input_file`.

                      - `"input_file"`

                    - `detail: optional "auto" or "low" or "high"`

                      The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                      - `"auto"`

                      - `"low"`

                      - `"high"`

                    - `file_data: optional string`

                      The content of the file to be sent to the model.

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `file_url: optional string`

                      The URL of the file to be sent to the model.

                    - `filename: optional string`

                      The name of the file to be sent to the model.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `phase: optional "commentary" or "final_answer" or null`

                Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
                For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
                phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

                - `"commentary"`

                - `"final_answer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                  - `input_audio: object { data, format }`

                    - `data: string`

                      Base64-encoded audio data.

                    - `format: "mp3" or "wav"`

                      The format of the audio data. Currently supported formats are `mp3` and
                      `wav`.

                      - `"mp3"`

                      - `"wav"`

                  - `type: "input_audio"`

                    The type of the input item. Always `input_audio`.

                    - `"input_audio"`

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

                  - `TextInput = string`

                    A text input to the model.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                  - `OutputText object { text, type }`

                    A text output from the model.

                    - `text: string`

                      The text output from the model.

                    - `type: "output_text"`

                      The type of the output text. Always `output_text`.

                      - `"output_text"`

                  - `InputImage object { image_url, type, detail }`

                    An image input block used within EvalItem content arrays.

                    - `image_url: string`

                      The URL of the image input.

                    - `type: "input_image"`

                      The type of the image input. Always `input_image`.

                      - `"input_image"`

                    - `detail: optional string`

                      The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                  - `ResponseInputAudio object { input_audio, type }`

                    An audio input to the model.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `ItemReferenceInputMessages object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

          - `"none"`

          - `"minimal"`

          - `"low"`

          - `"medium"`

          - `"high"`

          - `"xhigh"`

          - `"max"`

        - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

          An object specifying the format that the model must output.

          Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
          Structured Outputs which ensures the model will match your supplied JSON
          schema. Learn more in the [Structured Outputs
          guide](/api/docs/guides/structured-outputs).

          Setting to `{ "type": "json_object" }` enables the older JSON mode, which
          ensures the message the model generates is valid JSON. Using `json_schema`
          is preferred for models that support it.

          - `ResponseFormatText object { type }`

            Default response format. Used to generate text responses.

            - `type: "text"`

              The type of response format being defined. Always `text`.

              - `"text"`

          - `ResponseFormatJSONSchema object { json_schema, type }`

            JSON Schema response format. Used to generate structured JSON responses.
            Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

            - `json_schema: object { name, description, schema, strict }`

              Structured Outputs configuration options, including a JSON Schema.

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `schema: optional map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `type: "json_schema"`

              The type of response format being defined. Always `json_schema`.

              - `"json_schema"`

          - `ResponseFormatJSONObject object { type }`

            JSON object response format. An older method of generating JSON responses.
            Using `json_schema` is recommended for models that support it. Note that the
            model will not generate JSON without a system or user message instructing it
            to do so.

            - `type: "json_object"`

              The type of response format being defined. Always `json_object`.

              - `"json_object"`

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `tools: optional array of ChatCompletionFunctionTool`

          A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

          - `function: FunctionDefinition`

            - `name: string`

              The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

            - `description: optional string`

              A description of what the function does, used by the model to choose when and how to call the function.

            - `parameters: optional FunctionParameters`

              The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

              Omitting `parameters` defines a function with an empty parameter list.

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

          - `type: "function"`

            The type of the tool. Currently, only `function` is supported.

            - `"function"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

    - `ResponsesRunDataSource object { source, type, input_messages, 2 more }`

      A ResponsesRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or EvalResponsesSource { type, created_after, created_before, 8 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `EvalResponsesSource object { type, created_after, created_before, 8 more }`

          A EvalResponsesSource object describing a run data source configuration.

          - `type: "responses"`

            The type of run data source. Always `responses`.

            - `"responses"`

          - `created_after: optional number or null`

            Only include items created after this timestamp (inclusive). This is a query parameter used to select responses.

          - `created_before: optional number or null`

            Only include items created before this timestamp (inclusive). This is a query parameter used to select responses.

          - `instructions_search: optional string or null`

            Optional string to search the 'instructions' field. This is a query parameter used to select responses.

          - `metadata: optional unknown or null`

            Metadata filter for the responses. This is a query parameter used to select responses.

          - `model: optional string or null`

            The name of the model to find responses for. This is a query parameter used to select responses.

          - `reasoning_effort: optional ReasoningEffort or null`

            Constrains effort on reasoning for reasoning models. Currently supported
            values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
            Reducing reasoning effort can result in faster responses and fewer tokens
            used on reasoning in a response. Not all reasoning models support every
            value. See the
            [reasoning guide](/api/docs/guides/reasoning)
            for model-specific support.

          - `temperature: optional number or null`

            Sampling temperature. This is a query parameter used to select responses.

          - `tools: optional array of string or null`

            List of tool names. This is a query parameter used to select responses.

          - `top_p: optional number or null`

            Nucleus sampling parameter. This is a query parameter used to select responses.

          - `users: optional array of string or null`

            List of user identifiers. This is a query parameter used to select responses.

      - `type: "responses"`

        The type of run data source. Always `responses`.

        - `"responses"`

      - `input_messages: optional InputMessagesTemplate { template, type }  or InputMessagesItemReference { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `InputMessagesTemplate object { template, type }`

          - `template: array of ChatMessage { content, role }  or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `ChatMessage object { content, role }`

              - `content: string`

                The content of the message.

              - `role: string`

                The role of the message (e.g. "system", "assistant", "user").

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `InputMessagesItemReference object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.name"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `text: optional object { format }`

          Configuration options for a text response from the model. Can be plain
          text or structured JSON data. Learn more:

          - [Text inputs and outputs](/api/docs/guides/text)
          - [Structured Outputs](/api/docs/guides/structured-outputs)

          - `format: optional ResponseFormatTextConfig`

            An object specifying the format that the model must output.

            Configuring `{ "type": "json_schema" }` enables Structured Outputs,
            which ensures the model will match your supplied JSON schema. Learn more in the
            [Structured Outputs guide](/api/docs/guides/structured-outputs).

            The default format is `{ "type": "text" }` with no additional options.

            **Not recommended for gpt-4o and newer models:**

            Setting to `{ "type": "json_object" }` enables the older JSON mode, which
            ensures the message the model generates is valid JSON. Using `json_schema`
            is preferred for models that support it.

            - `ResponseFormatText object { type }`

              Default response format. Used to generate text responses.

            - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }`

              JSON Schema response format. Used to generate structured JSON responses.
              Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `schema: map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `type: "json_schema"`

                The type of response format being defined. Always `json_schema`.

                - `"json_schema"`

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `ResponseFormatJSONObject object { type }`

              JSON object response format. An older method of generating JSON responses.
              Using `json_schema` is recommended for models that support it. Note that the
              model will not generate JSON without a system or user message instructing it
              to do so.

        - `tools: optional array of Function { name, parameters, strict, 6 more }  or FileSearch { type, vector_store_ids, filters, 2 more }  or Computer { type }  or 13 more`

          An array of tools the model may call while generating a response. You
          can specify which tool to use by setting the `tool_choice` parameter.

          The two categories of tools you can provide the model are:

          - **Built-in tools**: Tools that are provided by OpenAI that extend the
            model's capabilities, like [web search](/api/docs/guides/tools-web-search)
            or [file search](/api/docs/guides/tools-file-search). Learn more about
            [built-in tools](/api/docs/guides/tools).
          - **Function calls (custom tools)**: Functions that are defined by you,
            enabling the model to call your own code. Learn more about
            [function calling](/api/docs/guides/function-calling).

          - `Function object { name, parameters, strict, 6 more }`

            Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling).

            - `name: string`

              The name of the function to call.

            - `parameters: map[unknown] or null`

              A JSON schema object describing the parameters of the function.

            - `strict: boolean or null`

              Whether strict parameter validation is enforced for this function tool.

            - `type: "function"`

              The type of the function tool. Always `function`.

              - `"function"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

            - `defer_loading: optional boolean`

              Whether this function is deferred and loaded via tool search.

            - `description: optional string or null`

              A description of the function. Used by the model to determine whether or not to call the function.

            - `output_schema: optional map[unknown] or null`

              A JSON schema object describing the JSON value encoded in string outputs for this function.

          - `FileSearch object { type, vector_store_ids, filters, 2 more }`

            A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search).

            - `type: "file_search"`

              The type of the file search tool. Always `file_search`.

              - `"file_search"`

            - `vector_store_ids: array of string`

              The IDs of the vector stores to search.

            - `filters: optional ComparisonFilter or CompoundFilter or null`

              A filter to apply.

              - `ComparisonFilter object { key, type, value }`

                A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                - `key: string`

                  The key to compare against the value.

                - `type: "eq" or "ne" or "gt" or 5 more`

                  Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`.

                  - `eq`: equals
                  - `ne`: not equal
                  - `gt`: greater than
                  - `gte`: greater than or equal
                  - `lt`: less than
                  - `lte`: less than or equal
                  - `in`: in
                  - `nin`: not in

                  - `"eq"`

                  - `"ne"`

                  - `"gt"`

                  - `"gte"`

                  - `"lt"`

                  - `"lte"`

                  - `"in"`

                  - `"nin"`

                - `value: string or number or boolean or array of string or number`

                  The value to compare against the attribute key; supports string, number, or boolean types.

                  - `string`

                  - `number`

                  - `boolean`

                  - `array of string or number`

                    - `string`

                    - `number`

              - `CompoundFilter object { filters, type }`

                Combine multiple filters using `and` or `or`.

                - `filters: array of ComparisonFilter or CompoundFilter`

                  Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.

                  - `ComparisonFilter object { key, type, value }`

                    A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                  - `CompoundFilter object { filters, type }`

                    Combine multiple filters using `and` or `or`.

                - `type: "and" or "or"`

                  Type of operation: `and` or `or`.

                  - `"and"`

                  - `"or"`

            - `max_num_results: optional number`

              The maximum number of results to return. This number should be between 1 and 50 inclusive.

            - `ranking_options: optional object { hybrid_search, ranker, score_threshold }`

              Ranking options for search.

              - `hybrid_search: optional object { embedding_weight, text_weight }`

                Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

                - `embedding_weight: number`

                  The weight of the embedding in the reciprocal ranking fusion.

                - `text_weight: number`

                  The weight of the text in the reciprocal ranking fusion.

              - `ranker: optional "auto" or "default-2024-11-15"`

                The ranker to use for the file search.

                - `"auto"`

                - `"default-2024-11-15"`

              - `score_threshold: optional number`

                The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

          - `Computer object { type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `type: "computer"`

              The type of the computer tool. Always `computer`.

              - `"computer"`

          - `ComputerUsePreview object { display_height, display_width, environment, type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `display_height: number`

              The height of the computer display.

            - `display_width: number`

              The width of the computer display.

            - `environment: "windows" or "mac" or "linux" or 2 more`

              The type of computer environment to control.

              - `"windows"`

              - `"mac"`

              - `"linux"`

              - `"ubuntu"`

              - `"browser"`

            - `type: "computer_use_preview"`

              The type of the computer use tool. Always `computer_use_preview`.

              - `"computer_use_preview"`

          - `WebSearch object { type, external_web_access, filters, 2 more }`

            Search the Internet for sources related to the prompt. Learn more about the
            [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search" or "web_search_2025_08_26"`

              The type of the web search tool. One of `web_search` or `web_search_2025_08_26`.

              - `"web_search"`

              - `"web_search_2025_08_26"`

            - `external_web_access: optional boolean`

              Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

            - `filters: optional object { allowed_domains }  or null`

              Filters for the search.

              - `allowed_domains: optional array of string or null`

                Allowed domains for the search. If not provided, all domains are allowed.
                Subdomains of the provided domains are allowed as well.

                Example: `["pubmed.ncbi.nlm.nih.gov"]`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { city, country, region, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the
              United States. To avoid this fallback, pass `{"type": "approximate"}` without
              location fields. To localize results, provide the relevant location fields.

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

              - `type: optional "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

          - `Mcp object { server_label, type, allowed_callers, 9 more }`

            Give the model access to additional tools via remote Model Context Protocol
            (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp).

            - `server_label: string`

              A label for this MCP server, used to identify it in tool calls.

            - `type: "mcp"`

              The type of the MCP tool. Always `mcp`.

              - `"mcp"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names }  or null`

              List of allowed tool names or a filter object.

              - `McpAllowedTools = array of string`

                A string array of allowed tool names

              - `McpToolFilter object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

            - `authorization: optional string`

              An OAuth access token that can be used with a remote MCP server, either
              with a custom MCP server URL or a service connector. Your application
              must handle the OAuth authorization flow and provide the token here.

            - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more`

              Identifier for service connectors, like those available in ChatGPT. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more
              about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors).

              This field is deprecated for models released after September 1, 2026.
              Use `server_url` to connect to a remote MCP server, or `tunnel_id` to
              connect through a Secure MCP Tunnel.

              Currently supported `connector_id` values are:

              - Dropbox: `connector_dropbox`
              - Gmail: `connector_gmail`
              - Google Calendar: `connector_googlecalendar`
              - Google Drive: `connector_googledrive`
              - Microsoft Teams: `connector_microsoftteams`
              - Outlook Calendar: `connector_outlookcalendar`
              - Outlook Email: `connector_outlookemail`
              - SharePoint: `connector_sharepoint`

              - `"connector_dropbox"`

              - `"connector_gmail"`

              - `"connector_googlecalendar"`

              - `"connector_googledrive"`

              - `"connector_microsoftteams"`

              - `"connector_outlookcalendar"`

              - `"connector_outlookemail"`

              - `"connector_sharepoint"`

            - `defer_loading: optional boolean`

              Whether this MCP tool is deferred and discovered via tool search.

            - `headers: optional map[string] or null`

              Optional HTTP headers to send to the MCP server. Use for authentication
              or other purposes.

            - `require_approval: optional McpToolApprovalFilter { always, never }  or "always" or "never" or null`

              Specify which of the MCP server's tools require approval.

              - `McpToolApprovalFilter object { always, never }`

                Specify which of the MCP server's tools require approval. Can be
                `always`, `never`, or a filter object associated with tools
                that require approval.

                - `always: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

                - `never: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

              - `McpToolApprovalSetting = "always" or "never"`

                Specify a single approval policy for all tools. One of `always` or
                `never`. When set to `always`, all tools will require approval. When
                set to `never`, all tools will not require approval.

                - `"always"`

                - `"never"`

            - `server_description: optional string`

              Optional description of the MCP server, used to provide more context.

            - `server_url: optional string`

              The URL for the MCP server. One of `server_url`, `connector_id`, or
              `tunnel_id` must be provided.

            - `tunnel_id: optional string`

              The Secure MCP Tunnel ID to use instead of a direct server URL. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided.

          - `CodeInterpreter object { container, type, allowed_callers }`

            A tool that runs Python code to help generate a response to a prompt.

            - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }`

              The code interpreter container. Can be a container ID or an object that
              specifies uploaded file IDs to make available to your code, along with an
              optional `memory_limit` setting.

              - `string`

                The container ID.

              - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }`

                Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

                - `type: "auto"`

                  Always `auto`.

                  - `"auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the code interpreter container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                    - `type: "disabled"`

                      Disable outbound network access. Always `disabled`.

                      - `"disabled"`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                    - `allowed_domains: array of string`

                      A list of allowed domains when type is `allowlist`.

                    - `type: "allowlist"`

                      Allow outbound network access only to specified domains. Always `allowlist`.

                      - `"allowlist"`

                    - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret`

                      Optional domain-scoped secrets for allowlisted domains.

                      - `domain: string`

                        The domain associated with the secret.

                      - `name: string`

                        The name of the secret to inject for the domain.

                      - `value: string`

                        The secret value to inject for the domain.

            - `type: "code_interpreter"`

              The type of the code interpreter tool. Always `code_interpreter`.

              - `"code_interpreter"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

          - `ProgrammaticToolCalling object { type }`

            - `type: "programmatic_tool_calling"`

              The type of the tool. Always `programmatic_tool_calling`.

              - `"programmatic_tool_calling"`

          - `ImageGeneration object { type, action, background, 9 more }`

            A tool that generates images using the GPT image models.

            - `type: "image_generation"`

              The type of the image generation tool. Always `image_generation`.

              - `"image_generation"`

            - `action: optional "generate" or "edit" or "auto"`

              Whether to generate a new image or edit an existing image. Default: `auto`.

              - `"generate"`

              - `"edit"`

              - `"auto"`

            - `background: optional "transparent" or "opaque" or "auto"`

              Set the background of the generated image. One of `transparent`, `opaque`,
              or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including
              their `2026-09-08` snapshots, support `opaque` and `transparent`
              backgrounds. Transparent backgrounds are available for supported GPT Image
              models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in
              preview. When using `transparent`, set the output format to `png` or `webp`.
              Default: `auto`.

              - `"transparent"`

              - `"opaque"`

              - `"auto"`

            - `input_fidelity: optional "high" or "low" or null`

              Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models.

              - `"high"`

              - `"low"`

            - `input_image_mask: optional object { file_id, image_url }`

              Optional mask for inpainting. Contains `image_url`
              (string, optional) and `file_id` (string, optional).

              - `file_id: optional string`

                File ID for the mask image.

              - `image_url: optional string`

                Base64-encoded mask image.

            - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

              The image generation model to use. One of `gpt-image-1`,
              `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
              `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
              `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
              `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
              `gpt-image-1`.

              - `string`

              - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

                The image generation model to use. One of `gpt-image-1`,
                `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
                `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
                `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
                `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
                `gpt-image-1`.

                - `"gpt-image-1"`

                - `"gpt-image-1-mini"`

                - `"gpt-image-1.5"`

                - `"gpt-image-2"`

                - `"gpt-image-2-2026-04-21"`

                - `"gpt-image-2.5-sunburst"`

                - `"gpt-image-2.5-sunburst-2026-09-08"`

                - `"gpt-image-2.5-flare"`

                - `"gpt-image-2.5-flare-2026-09-08"`

            - `moderation: optional "auto" or "low"`

              Moderation level for the generated image. Default: `auto`.

              - `"auto"`

              - `"low"`

            - `output_compression: optional number`

              Compression level for the output image. Default: 100.

            - `output_format: optional "png" or "webp" or "jpeg"`

              The output format of the generated image. One of `png`, `webp`, or
              `jpeg`. Default: `png`.

              - `"png"`

              - `"webp"`

              - `"jpeg"`

            - `partial_images: optional number`

              Number of partial images to generate in streaming mode, from 0 (default value) to 3.

            - `quality: optional "low" or "medium" or "high" or 3 more`

              The quality of the generated image. The GPT image models support `low`,
              `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`,
              including their `2026-09-08` snapshots, also support `xhigh` and `max`.
              Default: `auto`.

              - `"low"`

              - `"medium"`

              - `"high"`

              - `"xhigh"`

              - `"max"`

              - `"auto"`

            - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"`

              The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

              - `string`

              - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"`

                The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

                - `"1024x1024"`

                - `"1024x1536"`

                - `"1536x1024"`

                - `"auto"`

          - `LocalShell object { type }`

            A tool that allows the model to execute shell commands in a local environment.

            - `type: "local_shell"`

              The type of the local shell tool. Always `local_shell`.

              - `"local_shell"`

          - `Shell object { type, allowed_callers, environment }`

            A tool that allows the model to execute shell commands.

            - `type: "shell"`

              The type of the shell tool. Always `shell`.

              - `"shell"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null`

              - `ContainerAuto object { type, file_ids, memory_limit, 2 more }`

                - `type: "container_auto"`

                  Automatically creates a container for this request

                  - `"container_auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                - `skills: optional array of SkillReference or InlineSkill`

                  An optional list of skills referenced by id or inline data.

                  - `SkillReference object { skill_id, type, version }`

                    - `skill_id: string`

                      The ID of the referenced skill.

                    - `type: "skill_reference"`

                      References a skill created with the /v1/skills endpoint.

                      - `"skill_reference"`

                    - `version: optional string`

                      Optional skill version. Use a positive integer or 'latest'. Omit for default.

                  - `InlineSkill object { description, name, source, type }`

                    - `description: string`

                      The description of the skill.

                    - `name: string`

                      The name of the skill.

                    - `source: InlineSkillSource`

                      Inline skill payload

                      - `data: string`

                        Base64-encoded skill zip bundle.

                      - `media_type: "application/zip"`

                        The media type of the inline skill payload. Must be `application/zip`.

                        - `"application/zip"`

                      - `type: "base64"`

                        The type of the inline skill source. Must be `base64`.

                        - `"base64"`

                    - `type: "inline"`

                      Defines an inline skill for this request.

                      - `"inline"`

              - `LocalEnvironment object { type, skills }`

                - `type: "local"`

                  Use a local computer environment.

                  - `"local"`

                - `skills: optional array of LocalSkill`

                  An optional list of skills.

                  - `description: string`

                    The description of the skill.

                  - `name: string`

                    The name of the skill.

                  - `path: string`

                    The path to the directory containing the skill.

              - `ContainerReference object { container_id, type }`

                - `container_id: string`

                  The ID of the referenced container.

                - `type: "container_reference"`

                  References a container created with the /v1/containers endpoint

                  - `"container_reference"`

          - `Custom object { name, type, allowed_callers, 4 more }`

            A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

            - `name: string`

              The name of the custom tool, used to identify it in tool calls.

            - `type: "custom"`

              The type of the custom tool. Always `custom`.

              - `"custom"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

              Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

            - `defer_loading: optional boolean`

              Whether this tool should be deferred and discovered via tool search.

            - `description: optional string`

              Optional description of the custom tool, used to provide more context.

            - `format: optional CustomToolInputFormat`

              The input format for the custom tool. Default is unconstrained text.

              - `Text object { type }`

                Unconstrained free-form text.

                - `type: "text"`

                  Unconstrained text format. Always `text`.

                  - `"text"`

              - `Grammar object { definition, syntax, type }`

                A grammar defined by the user.

                - `definition: string`

                  The grammar definition.

                - `syntax: "lark" or "regex"`

                  The syntax of the grammar definition. One of `lark` or `regex`.

                  - `"lark"`

                  - `"regex"`

                - `type: "grammar"`

                  Grammar format. Always `grammar`.

                  - `"grammar"`

          - `Namespace object { description, name, tools, type }`

            Groups function/custom tools under a shared namespace.

            - `description: string`

              A description of the namespace shown to the model.

            - `name: string`

              The namespace name used in tool calls (for example, `crm`).

            - `tools: array of Function { name, type, allowed_callers, 6 more }  or Custom { name, type, allowed_callers, 4 more }`

              The function/custom tools available inside this namespace.

              - `Function object { name, type, allowed_callers, 6 more }`

                - `name: string`

                - `type: "function"`

                  - `"function"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this function should be deferred and discovered via tool search.

                - `description: optional string or null`

                - `output_schema: optional map[unknown] or null`

                  A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

                - `parameters: optional unknown or null`

                - `strict: optional boolean or null`

                  Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

              - `Custom object { name, type, allowed_callers, 4 more }`

                A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

                - `name: string`

                  The name of the custom tool, used to identify it in tool calls.

                - `type: "custom"`

                  The type of the custom tool. Always `custom`.

                  - `"custom"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this tool should be deferred and discovered via tool search.

                - `description: optional string`

                  Optional description of the custom tool, used to provide more context.

                - `format: optional CustomToolInputFormat`

                  The input format for the custom tool. Default is unconstrained text.

            - `type: "namespace"`

              The type of the tool. Always `namespace`.

              - `"namespace"`

          - `ToolSearch object { type, description, execution, parameters }`

            Hosted or BYOT tool search configuration for deferred tools.

            - `type: "tool_search"`

              The type of the tool. Always `tool_search`.

              - `"tool_search"`

            - `description: optional string or null`

              Description shown to the model for a client-executed tool search tool.

            - `execution: optional "server" or "client"`

              Whether tool search is executed by the server or by the client.

              - `"server"`

              - `"client"`

            - `parameters: optional unknown or null`

              Parameter schema for a client-executed tool search tool.

          - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }`

            This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search_preview" or "web_search_preview_2025_03_11"`

              The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`.

              - `"web_search_preview"`

              - `"web_search_preview_2025_03_11"`

            - `search_content_types: optional array of "text" or "image"`

              - `"text"`

              - `"image"`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { type, city, country, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields.

              - `type: "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

          - `ApplyPatch object { type, allowed_callers }`

            Allows the assistant to create, delete, or update files using unified diffs.

            - `type: "apply_patch"`

              The type of the tool. Always `apply_patch`.

              - `"apply_patch"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

  - `error: EvalAPIError or null`

    An object representing an error response from the Eval API.

    - `code: string`

      The error code.

    - `message: string`

      The error message.

  - `eval_id: string`

    The identifier of the associated evaluation.

  - `metadata: Metadata or null`

    Set of 16 key-value pairs that can be attached to an object. This can be
    useful for storing additional information about the object in a structured
    format, and querying for objects via API or the dashboard.

    Keys are strings with a maximum length of 64 characters. Values are strings
    with a maximum length of 512 characters.

  - `model: string or null`

    The model that is evaluated, if applicable.

  - `name: string or null`

    The name of the evaluation run.

  - `object: "eval.run"`

    The type of the object. Always "eval.run".

    - `"eval.run"`

  - `per_model_usage: array of object { cached_tokens, completion_tokens, invocation_count, 3 more }  or null`

    Usage statistics for each model during the evaluation run.

    - `cached_tokens: number`

      The number of tokens retrieved from cache.

    - `completion_tokens: number`

      The number of completion tokens generated.

    - `invocation_count: number`

      The number of invocations.

    - `model_name: string`

      The name of the model.

    - `prompt_tokens: number`

      The number of prompt tokens used.

    - `total_tokens: number`

      The total number of tokens used.

  - `per_testing_criteria_results: array of object { failed, passed, testing_criteria }  or null`

    Results per testing criteria applied during the evaluation run.

    - `failed: number`

      Number of tests failed for this criteria.

    - `passed: number`

      Number of tests passed for this criteria.

    - `testing_criteria: string`

      A description of the testing criteria.

  - `report_url: string`

    The URL to the rendered evaluation run report on the UI dashboard.

  - `result_counts: object { errored, failed, passed, total }`

    Counters summarizing the outcomes of the evaluation run.

    - `errored: number`

      Number of output items that resulted in an error.

    - `failed: number`

      Number of output items that failed to pass the evaluation.

    - `passed: number`

      Number of output items that passed the evaluation.

    - `total: number`

      Total number of executed output items.

  - `status: string`

    The status of the evaluation run.

- `first_id: string`

  The identifier of the first eval run in the data array.

- `has_more: boolean`

  Indicates whether there are more evals available.

- `last_id: string`

  The identifier of the last eval run in the data array.

- `object: "list"`

  The type of this object. It is always set to "list".

  - `"list"`

### Example

```http
curl https://api.openai.com/v1/evals/$EVAL_ID/runs \
    -H "Authorization: Bearer $OPENAI_API_KEY"
```

#### Response

```json
{
  "data": [
    {
      "id": "id",
      "created_at": 0,
      "data_source": {
        "source": {
          "content": [
            {
              "item": {
                "foo": "bar"
              },
              "sample": {
                "foo": "bar"
              }
            }
          ],
          "type": "file_content"
        },
        "type": "jsonl"
      },
      "error": {
        "code": "code",
        "message": "message"
      },
      "eval_id": "eval_id",
      "metadata": {
        "foo": "string"
      },
      "model": "model",
      "name": "name",
      "object": "eval.run",
      "per_model_usage": [
        {
          "cached_tokens": 0,
          "completion_tokens": 0,
          "invocation_count": 0,
          "model_name": "model_name",
          "prompt_tokens": 0,
          "total_tokens": 0
        }
      ],
      "per_testing_criteria_results": [
        {
          "failed": 0,
          "passed": 0,
          "testing_criteria": "testing_criteria"
        }
      ],
      "report_url": "https://example.com",
      "result_counts": {
        "errored": 0,
        "failed": 0,
        "passed": 0,
        "total": 0
      },
      "status": "status"
    }
  ],
  "first_id": "first_id",
  "has_more": true,
  "last_id": "last_id",
  "object": "list"
}
```

### Example

```http
curl https://api.openai.com/v1/evals/egroup_67abd54d9b0081909a86353f6fb9317a/runs \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json"
```

#### Response

```json
{
  "object": "list",
  "data": [
    {
      "object": "eval.run",
      "id": "evalrun_67e0c7d31560819090d60c0780591042",
      "eval_id": "eval_67e0c726d560819083f19a957c4c640b",
      "report_url": "https://platform.openai.com/evaluations/eval_67e0c726d560819083f19a957c4c640b",
      "status": "completed",
      "model": "o3-mini",
      "name": "bulk_with_negative_examples_o3-mini",
      "created_at": 1742784467,
      "result_counts": {
        "total": 1,
        "errored": 0,
        "failed": 0,
        "passed": 1
      },
      "per_model_usage": [
        {
          "model_name": "o3-mini",
          "invocation_count": 1,
          "prompt_tokens": 563,
          "completion_tokens": 874,
          "total_tokens": 1437,
          "cached_tokens": 0
        }
      ],
      "per_testing_criteria_results": [
        {
          "testing_criteria": "Push Notification Summary Grader-1808cd0b-eeec-4e0b-a519-337e79f4f5d1",
          "passed": 1,
          "failed": 0
        }
      ],
      "data_source": {
        "type": "completions",
        "source": {
          "type": "file_content",
          "content": [
            {
              "item": {
                "notifications": "\n- New message from Sarah: \"Can you call me later?\"\n- Your package has been delivered!\n- Flash sale: 20% off electronics for the next 2 hours!\n"
              }
            }
          ]
        },
        "input_messages": {
          "type": "template",
          "template": [
            {
              "type": "message",
              "role": "developer",
              "content": {
                "type": "input_text",
                "text": "\n\n\n\nYou are a helpful assistant that takes in an array of push notifications and returns a collapsed summary of them.\nThe push notification will be provided as follows:\n<push_notifications>\n...notificationlist...\n</push_notifications>\n\nYou should return just the summary and nothing else.\n\n\nYou should return a summary that is concise and snappy.\n\n\nHere is an example of a good summary:\n<push_notifications>\n- Traffic alert: Accident reported on Main Street.- Package out for delivery: Expected by 5 PM.- New friend suggestion: Connect with Emma.\n</push_notifications>\n<summary>\nTraffic alert, package expected by 5pm, suggestion for new friend (Emily).\n</summary>\n\n\nHere is an example of a bad summary:\n<push_notifications>\n- Traffic alert: Accident reported on Main Street.- Package out for delivery: Expected by 5 PM.- New friend suggestion: Connect with Emma.\n</push_notifications>\n<summary>\nTraffic alert reported on main street. You have a package that will arrive by 5pm, Emily is a new friend suggested for you.\n</summary>\n"
              }
            },
            {
              "type": "message",
              "role": "user",
              "content": {
                "type": "input_text",
                "text": "<push_notifications>{{item.notifications}}</push_notifications>"
              }
            }
          ]
        },
        "model": "o3-mini"
      },
      "error": null,
      "metadata": {}
    }
  ],
  "first_id": "evalrun_67e0c7d31560819090d60c0780591042",
  "last_id": "evalrun_67e0c7d31560819090d60c0780591042",
  "has_more": true
}
```

## Get an eval run

**get** `/evals/{eval_id}/runs/{run_id}`

Get an evaluation run by ID.

### Path Parameters

- `eval_id: string`

- `run_id: string`

### Returns

- `id: string`

  Unique identifier for the evaluation run.

- `created_at: number`

  Unix timestamp (in seconds) when the evaluation run was created.

- `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }`

  Information about the run's data source.

  - `CreateEvalJSONLRunDataSource object { source, type }`

    A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

      Determines what populates the `item` namespace in the data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

    - `type: "jsonl"`

      The type of data source. Always `jsonl`.

      - `"jsonl"`

  - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

    A CompletionsRunDataSource object describing a model sampling configuration.

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

      Determines what populates the `item` namespace in this run's data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

      - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

        A StoredCompletionsRunDataSource configuration describing a set of filters

        - `type: "stored_completions"`

          The type of source. Always `stored_completions`.

          - `"stored_completions"`

        - `created_after: optional number or null`

          An optional Unix timestamp to filter items created after this time.

        - `created_before: optional number or null`

          An optional Unix timestamp to filter items created before this time.

        - `limit: optional number or null`

          An optional maximum number of items to return.

        - `metadata: optional Metadata or null`

          Set of 16 key-value pairs that can be attached to an object. This can be
          useful for storing additional information about the object in a structured
          format, and querying for objects via API or the dashboard.

          Keys are strings with a maximum length of 64 characters. Values are strings
          with a maximum length of 512 characters.

        - `model: optional string or null`

          An optional model to filter by (e.g., 'gpt-6-astra').

    - `type: "completions"`

      The type of run data source. Always `completions`.

      - `"completions"`

    - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

      Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

      - `TemplateInputMessages object { template, type }`

        - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

          A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

          - `EasyInputMessage object { content, role, phase, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputMessageContentList`

              Text, image, or audio input to the model, used to generate a response.
              Can also contain previous assistant responses.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputMessageContentList = array of ResponseInputContent`

                A list of one or many input items to the model, containing different content
                types.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                  - `text: string`

                    The text input to the model.

                  - `type: "input_text"`

                    The type of the input item. Always `input_text`.

                    - `"input_text"`

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

                - `ResponseInputImage object { detail, type, file_id, 2 more }`

                  An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                  - `detail: ImageDetail`

                    The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                    - `"low"`

                    - `"high"`

                    - `"auto"`

                    - `"original"`

                  - `type: "input_image"`

                    The type of the input item. Always `input_image`.

                    - `"input_image"`

                  - `file_id: optional string or null`

                    The ID of the file to be sent to the model.

                  - `image_url: optional string or null`

                    The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

                - `ResponseInputFile object { type, detail, file_data, 4 more }`

                  A file input to the model.

                  - `type: "input_file"`

                    The type of the input item. Always `input_file`.

                    - `"input_file"`

                  - `detail: optional "auto" or "low" or "high"`

                    The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                    - `"auto"`

                    - `"low"`

                    - `"high"`

                  - `file_data: optional string`

                    The content of the file to be sent to the model.

                  - `file_id: optional string or null`

                    The ID of the file to be sent to the model.

                  - `file_url: optional string`

                    The URL of the file to be sent to the model.

                  - `filename: optional string`

                    The name of the file to be sent to the model.

                  - `prompt_cache_breakpoint: optional object { mode }`

                    Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                    - `mode: "explicit"`

                      The breakpoint mode. Always `explicit`.

                      - `"explicit"`

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `phase: optional "commentary" or "final_answer" or null`

              Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
              For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
              phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

              - `"commentary"`

              - `"final_answer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

          - `EvalMessageObject object { content, role, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

              Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

              - `OutputText object { text, type }`

                A text output from the model.

                - `text: string`

                  The text output from the model.

                - `type: "output_text"`

                  The type of the output text. Always `output_text`.

                  - `"output_text"`

              - `InputImage object { image_url, type, detail }`

                An image input block used within EvalItem content arrays.

                - `image_url: string`

                  The URL of the image input.

                - `type: "input_image"`

                  The type of the image input. Always `input_image`.

                  - `"input_image"`

                - `detail: optional string`

                  The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

              - `ResponseInputAudio object { input_audio, type }`

                An audio input to the model.

                - `input_audio: object { data, format }`

                  - `data: string`

                    Base64-encoded audio data.

                  - `format: "mp3" or "wav"`

                    The format of the audio data. Currently supported formats are `mp3` and
                    `wav`.

                    - `"mp3"`

                    - `"wav"`

                - `type: "input_audio"`

                  The type of the input item. Always `input_audio`.

                  - `"input_audio"`

              - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                A list of inputs, each of which may be either an input text, output text, input
                image, or input audio object.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

        - `type: "template"`

          The type of input messages. Always `template`.

          - `"template"`

      - `ItemReferenceInputMessages object { item_reference, type }`

        - `item_reference: string`

          A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

        - `type: "item_reference"`

          The type of input messages. Always `item_reference`.

          - `"item_reference"`

    - `model: optional string`

      The name of the model to use for generating completions (e.g. "o3-mini").

    - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

      - `max_completion_tokens: optional number`

        The maximum number of tokens in the generated output.

      - `reasoning_effort: optional ReasoningEffort or null`

        Constrains effort on reasoning for reasoning models. Currently supported
        values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
        Reducing reasoning effort can result in faster responses and fewer tokens
        used on reasoning in a response. Not all reasoning models support every
        value. See the
        [reasoning guide](/api/docs/guides/reasoning)
        for model-specific support.

        - `"none"`

        - `"minimal"`

        - `"low"`

        - `"medium"`

        - `"high"`

        - `"xhigh"`

        - `"max"`

      - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

        An object specifying the format that the model must output.

        Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
        Structured Outputs which ensures the model will match your supplied JSON
        schema. Learn more in the [Structured Outputs
        guide](/api/docs/guides/structured-outputs).

        Setting to `{ "type": "json_object" }` enables the older JSON mode, which
        ensures the message the model generates is valid JSON. Using `json_schema`
        is preferred for models that support it.

        - `ResponseFormatText object { type }`

          Default response format. Used to generate text responses.

          - `type: "text"`

            The type of response format being defined. Always `text`.

            - `"text"`

        - `ResponseFormatJSONSchema object { json_schema, type }`

          JSON Schema response format. Used to generate structured JSON responses.
          Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

          - `json_schema: object { name, description, schema, strict }`

            Structured Outputs configuration options, including a JSON Schema.

            - `name: string`

              The name of the response format. Must be a-z, A-Z, 0-9, or contain
              underscores and dashes, with a maximum length of 64.

            - `description: optional string`

              A description of what the response format is for, used by the model to
              determine how to respond in the format.

            - `schema: optional map[unknown]`

              The schema for the response format, described as a JSON Schema object.
              Learn how to build JSON schemas [here](https://json-schema.org/).

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the output.
              If set to true, the model will always follow the exact schema defined
              in the `schema` field. Only a subset of JSON Schema is supported when
              `strict` is `true`. To learn more, read the [Structured Outputs
              guide](/api/docs/guides/structured-outputs).

          - `type: "json_schema"`

            The type of response format being defined. Always `json_schema`.

            - `"json_schema"`

        - `ResponseFormatJSONObject object { type }`

          JSON object response format. An older method of generating JSON responses.
          Using `json_schema` is recommended for models that support it. Note that the
          model will not generate JSON without a system or user message instructing it
          to do so.

          - `type: "json_object"`

            The type of response format being defined. Always `json_object`.

            - `"json_object"`

      - `seed: optional number`

        A seed value to initialize the randomness, during sampling.

      - `temperature: optional number`

        A higher temperature increases randomness in the outputs.

      - `tools: optional array of ChatCompletionFunctionTool`

        A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

        - `function: FunctionDefinition`

          - `name: string`

            The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

          - `description: optional string`

            A description of what the function does, used by the model to choose when and how to call the function.

          - `parameters: optional FunctionParameters`

            The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

            Omitting `parameters` defines a function with an empty parameter list.

          - `strict: optional boolean or null`

            Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

        - `type: "function"`

          The type of the tool. Currently, only `function` is supported.

          - `"function"`

      - `top_p: optional number`

        An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

  - `ResponsesRunDataSource object { source, type, input_messages, 2 more }`

    A ResponsesRunDataSource object describing a model sampling configuration.

    - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or EvalResponsesSource { type, created_after, created_before, 8 more }`

      Determines what populates the `item` namespace in this run's data source.

      - `EvalJSONLFileContentSource object { content, type }`

        - `content: array of object { item, sample }`

          The content of the jsonl file.

          - `item: map[unknown]`

          - `sample: optional map[unknown]`

        - `type: "file_content"`

          The type of jsonl source. Always `file_content`.

          - `"file_content"`

      - `EvalJSONLFileIDSource object { id, type }`

        - `id: string`

          The identifier of the file.

        - `type: "file_id"`

          The type of jsonl source. Always `file_id`.

          - `"file_id"`

      - `EvalResponsesSource object { type, created_after, created_before, 8 more }`

        A EvalResponsesSource object describing a run data source configuration.

        - `type: "responses"`

          The type of run data source. Always `responses`.

          - `"responses"`

        - `created_after: optional number or null`

          Only include items created after this timestamp (inclusive). This is a query parameter used to select responses.

        - `created_before: optional number or null`

          Only include items created before this timestamp (inclusive). This is a query parameter used to select responses.

        - `instructions_search: optional string or null`

          Optional string to search the 'instructions' field. This is a query parameter used to select responses.

        - `metadata: optional unknown or null`

          Metadata filter for the responses. This is a query parameter used to select responses.

        - `model: optional string or null`

          The name of the model to find responses for. This is a query parameter used to select responses.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

        - `temperature: optional number or null`

          Sampling temperature. This is a query parameter used to select responses.

        - `tools: optional array of string or null`

          List of tool names. This is a query parameter used to select responses.

        - `top_p: optional number or null`

          Nucleus sampling parameter. This is a query parameter used to select responses.

        - `users: optional array of string or null`

          List of user identifiers. This is a query parameter used to select responses.

    - `type: "responses"`

      The type of run data source. Always `responses`.

      - `"responses"`

    - `input_messages: optional InputMessagesTemplate { template, type }  or InputMessagesItemReference { item_reference, type }`

      Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

      - `InputMessagesTemplate object { template, type }`

        - `template: array of ChatMessage { content, role }  or EvalMessageObject { content, role, type }`

          A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

          - `ChatMessage object { content, role }`

            - `content: string`

              The content of the message.

            - `role: string`

              The role of the message (e.g. "system", "assistant", "user").

          - `EvalMessageObject object { content, role, type }`

            A message input to the model with a role indicating instruction following
            hierarchy. Instructions given with the `developer` or `system` role take
            precedence over instructions given with the `user` role. Messages with the
            `assistant` role are presumed to have been generated by the model in previous
            interactions.

            - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

              Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

              - `OutputText object { text, type }`

                A text output from the model.

                - `text: string`

                  The text output from the model.

                - `type: "output_text"`

                  The type of the output text. Always `output_text`.

                  - `"output_text"`

              - `InputImage object { image_url, type, detail }`

                An image input block used within EvalItem content arrays.

                - `image_url: string`

                  The URL of the image input.

                - `type: "input_image"`

                  The type of the image input. Always `input_image`.

                  - `"input_image"`

                - `detail: optional string`

                  The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

              - `ResponseInputAudio object { input_audio, type }`

                An audio input to the model.

              - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                A list of inputs, each of which may be either an input text, output text, input
                image, or input audio object.

            - `role: "user" or "assistant" or "system" or "developer"`

              The role of the message input. One of `user`, `assistant`, `system`, or
              `developer`.

              - `"user"`

              - `"assistant"`

              - `"system"`

              - `"developer"`

            - `type: optional "message"`

              The type of the message input. Always `message`.

              - `"message"`

        - `type: "template"`

          The type of input messages. Always `template`.

          - `"template"`

      - `InputMessagesItemReference object { item_reference, type }`

        - `item_reference: string`

          A reference to a variable in the `item` namespace. Ie, "item.name"

        - `type: "item_reference"`

          The type of input messages. Always `item_reference`.

          - `"item_reference"`

    - `model: optional string`

      The name of the model to use for generating completions (e.g. "o3-mini").

    - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }`

      - `max_completion_tokens: optional number`

        The maximum number of tokens in the generated output.

      - `reasoning_effort: optional ReasoningEffort or null`

        Constrains effort on reasoning for reasoning models. Currently supported
        values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
        Reducing reasoning effort can result in faster responses and fewer tokens
        used on reasoning in a response. Not all reasoning models support every
        value. See the
        [reasoning guide](/api/docs/guides/reasoning)
        for model-specific support.

      - `seed: optional number`

        A seed value to initialize the randomness, during sampling.

      - `temperature: optional number`

        A higher temperature increases randomness in the outputs.

      - `text: optional object { format }`

        Configuration options for a text response from the model. Can be plain
        text or structured JSON data. Learn more:

        - [Text inputs and outputs](/api/docs/guides/text)
        - [Structured Outputs](/api/docs/guides/structured-outputs)

        - `format: optional ResponseFormatTextConfig`

          An object specifying the format that the model must output.

          Configuring `{ "type": "json_schema" }` enables Structured Outputs,
          which ensures the model will match your supplied JSON schema. Learn more in the
          [Structured Outputs guide](/api/docs/guides/structured-outputs).

          The default format is `{ "type": "text" }` with no additional options.

          **Not recommended for gpt-4o and newer models:**

          Setting to `{ "type": "json_object" }` enables the older JSON mode, which
          ensures the message the model generates is valid JSON. Using `json_schema`
          is preferred for models that support it.

          - `ResponseFormatText object { type }`

            Default response format. Used to generate text responses.

          - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }`

            JSON Schema response format. Used to generate structured JSON responses.
            Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

            - `name: string`

              The name of the response format. Must be a-z, A-Z, 0-9, or contain
              underscores and dashes, with a maximum length of 64.

            - `schema: map[unknown]`

              The schema for the response format, described as a JSON Schema object.
              Learn how to build JSON schemas [here](https://json-schema.org/).

            - `type: "json_schema"`

              The type of response format being defined. Always `json_schema`.

              - `"json_schema"`

            - `description: optional string`

              A description of what the response format is for, used by the model to
              determine how to respond in the format.

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the output.
              If set to true, the model will always follow the exact schema defined
              in the `schema` field. Only a subset of JSON Schema is supported when
              `strict` is `true`. To learn more, read the [Structured Outputs
              guide](/api/docs/guides/structured-outputs).

          - `ResponseFormatJSONObject object { type }`

            JSON object response format. An older method of generating JSON responses.
            Using `json_schema` is recommended for models that support it. Note that the
            model will not generate JSON without a system or user message instructing it
            to do so.

      - `tools: optional array of Function { name, parameters, strict, 6 more }  or FileSearch { type, vector_store_ids, filters, 2 more }  or Computer { type }  or 13 more`

        An array of tools the model may call while generating a response. You
        can specify which tool to use by setting the `tool_choice` parameter.

        The two categories of tools you can provide the model are:

        - **Built-in tools**: Tools that are provided by OpenAI that extend the
          model's capabilities, like [web search](/api/docs/guides/tools-web-search)
          or [file search](/api/docs/guides/tools-file-search). Learn more about
          [built-in tools](/api/docs/guides/tools).
        - **Function calls (custom tools)**: Functions that are defined by you,
          enabling the model to call your own code. Learn more about
          [function calling](/api/docs/guides/function-calling).

        - `Function object { name, parameters, strict, 6 more }`

          Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling).

          - `name: string`

            The name of the function to call.

          - `parameters: map[unknown] or null`

            A JSON schema object describing the parameters of the function.

          - `strict: boolean or null`

            Whether strict parameter validation is enforced for this function tool.

          - `type: "function"`

            The type of the function tool. Always `function`.

            - `"function"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `async: optional boolean`

          - `defer_loading: optional boolean`

            Whether this function is deferred and loaded via tool search.

          - `description: optional string or null`

            A description of the function. Used by the model to determine whether or not to call the function.

          - `output_schema: optional map[unknown] or null`

            A JSON schema object describing the JSON value encoded in string outputs for this function.

        - `FileSearch object { type, vector_store_ids, filters, 2 more }`

          A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search).

          - `type: "file_search"`

            The type of the file search tool. Always `file_search`.

            - `"file_search"`

          - `vector_store_ids: array of string`

            The IDs of the vector stores to search.

          - `filters: optional ComparisonFilter or CompoundFilter or null`

            A filter to apply.

            - `ComparisonFilter object { key, type, value }`

              A filter used to compare a specified attribute key to a given value using a defined comparison operation.

              - `key: string`

                The key to compare against the value.

              - `type: "eq" or "ne" or "gt" or 5 more`

                Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`.

                - `eq`: equals
                - `ne`: not equal
                - `gt`: greater than
                - `gte`: greater than or equal
                - `lt`: less than
                - `lte`: less than or equal
                - `in`: in
                - `nin`: not in

                - `"eq"`

                - `"ne"`

                - `"gt"`

                - `"gte"`

                - `"lt"`

                - `"lte"`

                - `"in"`

                - `"nin"`

              - `value: string or number or boolean or array of string or number`

                The value to compare against the attribute key; supports string, number, or boolean types.

                - `string`

                - `number`

                - `boolean`

                - `array of string or number`

                  - `string`

                  - `number`

            - `CompoundFilter object { filters, type }`

              Combine multiple filters using `and` or `or`.

              - `filters: array of ComparisonFilter or CompoundFilter`

                Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.

                - `ComparisonFilter object { key, type, value }`

                  A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                - `CompoundFilter object { filters, type }`

                  Combine multiple filters using `and` or `or`.

              - `type: "and" or "or"`

                Type of operation: `and` or `or`.

                - `"and"`

                - `"or"`

          - `max_num_results: optional number`

            The maximum number of results to return. This number should be between 1 and 50 inclusive.

          - `ranking_options: optional object { hybrid_search, ranker, score_threshold }`

            Ranking options for search.

            - `hybrid_search: optional object { embedding_weight, text_weight }`

              Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

              - `embedding_weight: number`

                The weight of the embedding in the reciprocal ranking fusion.

              - `text_weight: number`

                The weight of the text in the reciprocal ranking fusion.

            - `ranker: optional "auto" or "default-2024-11-15"`

              The ranker to use for the file search.

              - `"auto"`

              - `"default-2024-11-15"`

            - `score_threshold: optional number`

              The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

        - `Computer object { type }`

          A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

          - `type: "computer"`

            The type of the computer tool. Always `computer`.

            - `"computer"`

        - `ComputerUsePreview object { display_height, display_width, environment, type }`

          A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

          - `display_height: number`

            The height of the computer display.

          - `display_width: number`

            The width of the computer display.

          - `environment: "windows" or "mac" or "linux" or 2 more`

            The type of computer environment to control.

            - `"windows"`

            - `"mac"`

            - `"linux"`

            - `"ubuntu"`

            - `"browser"`

          - `type: "computer_use_preview"`

            The type of the computer use tool. Always `computer_use_preview`.

            - `"computer_use_preview"`

        - `WebSearch object { type, external_web_access, filters, 2 more }`

          Search the Internet for sources related to the prompt. Learn more about the
          [web search tool](/api/docs/guides/tools-web-search).

          - `type: "web_search" or "web_search_2025_08_26"`

            The type of the web search tool. One of `web_search` or `web_search_2025_08_26`.

            - `"web_search"`

            - `"web_search_2025_08_26"`

          - `external_web_access: optional boolean`

            Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

          - `filters: optional object { allowed_domains }  or null`

            Filters for the search.

            - `allowed_domains: optional array of string or null`

              Allowed domains for the search. If not provided, all domains are allowed.
              Subdomains of the provided domains are allowed as well.

              Example: `["pubmed.ncbi.nlm.nih.gov"]`

          - `search_context_size: optional "low" or "medium" or "high"`

            High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

            - `"low"`

            - `"medium"`

            - `"high"`

          - `user_location: optional object { city, country, region, 2 more }  or null`

            The approximate location of the user. If omitted or null, defaults to the
            United States. To avoid this fallback, pass `{"type": "approximate"}` without
            location fields. To localize results, provide the relevant location fields.

            - `city: optional string or null`

              Free text input for the city of the user, e.g. `San Francisco`.

            - `country: optional string or null`

              The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

            - `region: optional string or null`

              Free text input for the region of the user, e.g. `California`.

            - `timezone: optional string or null`

              The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

            - `type: optional "approximate"`

              The type of location approximation. Always `approximate`.

              - `"approximate"`

        - `Mcp object { server_label, type, allowed_callers, 9 more }`

          Give the model access to additional tools via remote Model Context Protocol
          (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp).

          - `server_label: string`

            A label for this MCP server, used to identify it in tool calls.

          - `type: "mcp"`

            The type of the MCP tool. Always `mcp`.

            - `"mcp"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names }  or null`

            List of allowed tool names or a filter object.

            - `McpAllowedTools = array of string`

              A string array of allowed tool names

            - `McpToolFilter object { read_only, tool_names }`

              A filter object to specify which tools are allowed.

              - `read_only: optional boolean`

                Indicates whether or not a tool modifies data or is read-only. If an
                MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                it will match this filter.

              - `tool_names: optional array of string`

                List of allowed tool names.

          - `authorization: optional string`

            An OAuth access token that can be used with a remote MCP server, either
            with a custom MCP server URL or a service connector. Your application
            must handle the OAuth authorization flow and provide the token here.

          - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more`

            Identifier for service connectors, like those available in ChatGPT. One of
            `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more
            about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors).

            This field is deprecated for models released after September 1, 2026.
            Use `server_url` to connect to a remote MCP server, or `tunnel_id` to
            connect through a Secure MCP Tunnel.

            Currently supported `connector_id` values are:

            - Dropbox: `connector_dropbox`
            - Gmail: `connector_gmail`
            - Google Calendar: `connector_googlecalendar`
            - Google Drive: `connector_googledrive`
            - Microsoft Teams: `connector_microsoftteams`
            - Outlook Calendar: `connector_outlookcalendar`
            - Outlook Email: `connector_outlookemail`
            - SharePoint: `connector_sharepoint`

            - `"connector_dropbox"`

            - `"connector_gmail"`

            - `"connector_googlecalendar"`

            - `"connector_googledrive"`

            - `"connector_microsoftteams"`

            - `"connector_outlookcalendar"`

            - `"connector_outlookemail"`

            - `"connector_sharepoint"`

          - `defer_loading: optional boolean`

            Whether this MCP tool is deferred and discovered via tool search.

          - `headers: optional map[string] or null`

            Optional HTTP headers to send to the MCP server. Use for authentication
            or other purposes.

          - `require_approval: optional McpToolApprovalFilter { always, never }  or "always" or "never" or null`

            Specify which of the MCP server's tools require approval.

            - `McpToolApprovalFilter object { always, never }`

              Specify which of the MCP server's tools require approval. Can be
              `always`, `never`, or a filter object associated with tools
              that require approval.

              - `always: optional object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

              - `never: optional object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

            - `McpToolApprovalSetting = "always" or "never"`

              Specify a single approval policy for all tools. One of `always` or
              `never`. When set to `always`, all tools will require approval. When
              set to `never`, all tools will not require approval.

              - `"always"`

              - `"never"`

          - `server_description: optional string`

            Optional description of the MCP server, used to provide more context.

          - `server_url: optional string`

            The URL for the MCP server. One of `server_url`, `connector_id`, or
            `tunnel_id` must be provided.

          - `tunnel_id: optional string`

            The Secure MCP Tunnel ID to use instead of a direct server URL. One of
            `server_url`, `connector_id`, or `tunnel_id` must be provided.

        - `CodeInterpreter object { container, type, allowed_callers }`

          A tool that runs Python code to help generate a response to a prompt.

          - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }`

            The code interpreter container. Can be a container ID or an object that
            specifies uploaded file IDs to make available to your code, along with an
            optional `memory_limit` setting.

            - `string`

              The container ID.

            - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }`

              Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

              - `type: "auto"`

                Always `auto`.

                - `"auto"`

              - `file_ids: optional array of string`

                An optional list of uploaded files to make available to your code.

              - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                The memory limit for the code interpreter container.

                - `"1g"`

                - `"4g"`

                - `"16g"`

                - `"64g"`

              - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                Network access policy for the container.

                - `ContainerNetworkPolicyDisabled object { type }`

                  - `type: "disabled"`

                    Disable outbound network access. Always `disabled`.

                    - `"disabled"`

                - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                  - `allowed_domains: array of string`

                    A list of allowed domains when type is `allowlist`.

                  - `type: "allowlist"`

                    Allow outbound network access only to specified domains. Always `allowlist`.

                    - `"allowlist"`

                  - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret`

                    Optional domain-scoped secrets for allowlisted domains.

                    - `domain: string`

                      The domain associated with the secret.

                    - `name: string`

                      The name of the secret to inject for the domain.

                    - `value: string`

                      The secret value to inject for the domain.

          - `type: "code_interpreter"`

            The type of the code interpreter tool. Always `code_interpreter`.

            - `"code_interpreter"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

        - `ProgrammaticToolCalling object { type }`

          - `type: "programmatic_tool_calling"`

            The type of the tool. Always `programmatic_tool_calling`.

            - `"programmatic_tool_calling"`

        - `ImageGeneration object { type, action, background, 9 more }`

          A tool that generates images using the GPT image models.

          - `type: "image_generation"`

            The type of the image generation tool. Always `image_generation`.

            - `"image_generation"`

          - `action: optional "generate" or "edit" or "auto"`

            Whether to generate a new image or edit an existing image. Default: `auto`.

            - `"generate"`

            - `"edit"`

            - `"auto"`

          - `background: optional "transparent" or "opaque" or "auto"`

            Set the background of the generated image. One of `transparent`, `opaque`,
            or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including
            their `2026-09-08` snapshots, support `opaque` and `transparent`
            backgrounds. Transparent backgrounds are available for supported GPT Image
            models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in
            preview. When using `transparent`, set the output format to `png` or `webp`.
            Default: `auto`.

            - `"transparent"`

            - `"opaque"`

            - `"auto"`

          - `input_fidelity: optional "high" or "low" or null`

            Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models.

            - `"high"`

            - `"low"`

          - `input_image_mask: optional object { file_id, image_url }`

            Optional mask for inpainting. Contains `image_url`
            (string, optional) and `file_id` (string, optional).

            - `file_id: optional string`

              File ID for the mask image.

            - `image_url: optional string`

              Base64-encoded mask image.

          - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

            The image generation model to use. One of `gpt-image-1`,
            `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
            `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
            `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
            `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
            `gpt-image-1`.

            - `string`

            - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

              The image generation model to use. One of `gpt-image-1`,
              `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
              `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
              `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
              `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
              `gpt-image-1`.

              - `"gpt-image-1"`

              - `"gpt-image-1-mini"`

              - `"gpt-image-1.5"`

              - `"gpt-image-2"`

              - `"gpt-image-2-2026-04-21"`

              - `"gpt-image-2.5-sunburst"`

              - `"gpt-image-2.5-sunburst-2026-09-08"`

              - `"gpt-image-2.5-flare"`

              - `"gpt-image-2.5-flare-2026-09-08"`

          - `moderation: optional "auto" or "low"`

            Moderation level for the generated image. Default: `auto`.

            - `"auto"`

            - `"low"`

          - `output_compression: optional number`

            Compression level for the output image. Default: 100.

          - `output_format: optional "png" or "webp" or "jpeg"`

            The output format of the generated image. One of `png`, `webp`, or
            `jpeg`. Default: `png`.

            - `"png"`

            - `"webp"`

            - `"jpeg"`

          - `partial_images: optional number`

            Number of partial images to generate in streaming mode, from 0 (default value) to 3.

          - `quality: optional "low" or "medium" or "high" or 3 more`

            The quality of the generated image. The GPT image models support `low`,
            `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`,
            including their `2026-09-08` snapshots, also support `xhigh` and `max`.
            Default: `auto`.

            - `"low"`

            - `"medium"`

            - `"high"`

            - `"xhigh"`

            - `"max"`

            - `"auto"`

          - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"`

            The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

            - `string`

            - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"`

              The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

              - `"1024x1024"`

              - `"1024x1536"`

              - `"1536x1024"`

              - `"auto"`

        - `LocalShell object { type }`

          A tool that allows the model to execute shell commands in a local environment.

          - `type: "local_shell"`

            The type of the local shell tool. Always `local_shell`.

            - `"local_shell"`

        - `Shell object { type, allowed_callers, environment }`

          A tool that allows the model to execute shell commands.

          - `type: "shell"`

            The type of the shell tool. Always `shell`.

            - `"shell"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null`

            - `ContainerAuto object { type, file_ids, memory_limit, 2 more }`

              - `type: "container_auto"`

                Automatically creates a container for this request

                - `"container_auto"`

              - `file_ids: optional array of string`

                An optional list of uploaded files to make available to your code.

              - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                The memory limit for the container.

                - `"1g"`

                - `"4g"`

                - `"16g"`

                - `"64g"`

              - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                Network access policy for the container.

                - `ContainerNetworkPolicyDisabled object { type }`

                - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

              - `skills: optional array of SkillReference or InlineSkill`

                An optional list of skills referenced by id or inline data.

                - `SkillReference object { skill_id, type, version }`

                  - `skill_id: string`

                    The ID of the referenced skill.

                  - `type: "skill_reference"`

                    References a skill created with the /v1/skills endpoint.

                    - `"skill_reference"`

                  - `version: optional string`

                    Optional skill version. Use a positive integer or 'latest'. Omit for default.

                - `InlineSkill object { description, name, source, type }`

                  - `description: string`

                    The description of the skill.

                  - `name: string`

                    The name of the skill.

                  - `source: InlineSkillSource`

                    Inline skill payload

                    - `data: string`

                      Base64-encoded skill zip bundle.

                    - `media_type: "application/zip"`

                      The media type of the inline skill payload. Must be `application/zip`.

                      - `"application/zip"`

                    - `type: "base64"`

                      The type of the inline skill source. Must be `base64`.

                      - `"base64"`

                  - `type: "inline"`

                    Defines an inline skill for this request.

                    - `"inline"`

            - `LocalEnvironment object { type, skills }`

              - `type: "local"`

                Use a local computer environment.

                - `"local"`

              - `skills: optional array of LocalSkill`

                An optional list of skills.

                - `description: string`

                  The description of the skill.

                - `name: string`

                  The name of the skill.

                - `path: string`

                  The path to the directory containing the skill.

            - `ContainerReference object { container_id, type }`

              - `container_id: string`

                The ID of the referenced container.

              - `type: "container_reference"`

                References a container created with the /v1/containers endpoint

                - `"container_reference"`

        - `Custom object { name, type, allowed_callers, 4 more }`

          A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

          - `name: string`

            The name of the custom tool, used to identify it in tool calls.

          - `type: "custom"`

            The type of the custom tool. Always `custom`.

            - `"custom"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

          - `async: optional boolean`

            Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

          - `defer_loading: optional boolean`

            Whether this tool should be deferred and discovered via tool search.

          - `description: optional string`

            Optional description of the custom tool, used to provide more context.

          - `format: optional CustomToolInputFormat`

            The input format for the custom tool. Default is unconstrained text.

            - `Text object { type }`

              Unconstrained free-form text.

              - `type: "text"`

                Unconstrained text format. Always `text`.

                - `"text"`

            - `Grammar object { definition, syntax, type }`

              A grammar defined by the user.

              - `definition: string`

                The grammar definition.

              - `syntax: "lark" or "regex"`

                The syntax of the grammar definition. One of `lark` or `regex`.

                - `"lark"`

                - `"regex"`

              - `type: "grammar"`

                Grammar format. Always `grammar`.

                - `"grammar"`

        - `Namespace object { description, name, tools, type }`

          Groups function/custom tools under a shared namespace.

          - `description: string`

            A description of the namespace shown to the model.

          - `name: string`

            The namespace name used in tool calls (for example, `crm`).

          - `tools: array of Function { name, type, allowed_callers, 6 more }  or Custom { name, type, allowed_callers, 4 more }`

            The function/custom tools available inside this namespace.

            - `Function object { name, type, allowed_callers, 6 more }`

              - `name: string`

              - `type: "function"`

                - `"function"`

              - `allowed_callers: optional array of "direct" or "programmatic" or null`

                The tool invocation context(s).

                - `"direct"`

                - `"programmatic"`

              - `async: optional boolean`

                Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

              - `defer_loading: optional boolean`

                Whether this function should be deferred and discovered via tool search.

              - `description: optional string or null`

              - `output_schema: optional map[unknown] or null`

                A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

              - `parameters: optional unknown or null`

              - `strict: optional boolean or null`

                Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

            - `Custom object { name, type, allowed_callers, 4 more }`

              A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

              - `name: string`

                The name of the custom tool, used to identify it in tool calls.

              - `type: "custom"`

                The type of the custom tool. Always `custom`.

                - `"custom"`

              - `allowed_callers: optional array of "direct" or "programmatic" or null`

                The tool invocation context(s).

                - `"direct"`

                - `"programmatic"`

              - `async: optional boolean`

                Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

              - `defer_loading: optional boolean`

                Whether this tool should be deferred and discovered via tool search.

              - `description: optional string`

                Optional description of the custom tool, used to provide more context.

              - `format: optional CustomToolInputFormat`

                The input format for the custom tool. Default is unconstrained text.

          - `type: "namespace"`

            The type of the tool. Always `namespace`.

            - `"namespace"`

        - `ToolSearch object { type, description, execution, parameters }`

          Hosted or BYOT tool search configuration for deferred tools.

          - `type: "tool_search"`

            The type of the tool. Always `tool_search`.

            - `"tool_search"`

          - `description: optional string or null`

            Description shown to the model for a client-executed tool search tool.

          - `execution: optional "server" or "client"`

            Whether tool search is executed by the server or by the client.

            - `"server"`

            - `"client"`

          - `parameters: optional unknown or null`

            Parameter schema for a client-executed tool search tool.

        - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }`

          This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search).

          - `type: "web_search_preview" or "web_search_preview_2025_03_11"`

            The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`.

            - `"web_search_preview"`

            - `"web_search_preview_2025_03_11"`

          - `search_content_types: optional array of "text" or "image"`

            - `"text"`

            - `"image"`

          - `search_context_size: optional "low" or "medium" or "high"`

            High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

            - `"low"`

            - `"medium"`

            - `"high"`

          - `user_location: optional object { type, city, country, 2 more }  or null`

            The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields.

            - `type: "approximate"`

              The type of location approximation. Always `approximate`.

              - `"approximate"`

            - `city: optional string or null`

              Free text input for the city of the user, e.g. `San Francisco`.

            - `country: optional string or null`

              The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

            - `region: optional string or null`

              Free text input for the region of the user, e.g. `California`.

            - `timezone: optional string or null`

              The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

        - `ApplyPatch object { type, allowed_callers }`

          Allows the assistant to create, delete, or update files using unified diffs.

          - `type: "apply_patch"`

            The type of the tool. Always `apply_patch`.

            - `"apply_patch"`

          - `allowed_callers: optional array of "direct" or "programmatic" or null`

            The tool invocation context(s).

            - `"direct"`

            - `"programmatic"`

      - `top_p: optional number`

        An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

- `error: EvalAPIError or null`

  An object representing an error response from the Eval API.

  - `code: string`

    The error code.

  - `message: string`

    The error message.

- `eval_id: string`

  The identifier of the associated evaluation.

- `metadata: Metadata or null`

  Set of 16 key-value pairs that can be attached to an object. This can be
  useful for storing additional information about the object in a structured
  format, and querying for objects via API or the dashboard.

  Keys are strings with a maximum length of 64 characters. Values are strings
  with a maximum length of 512 characters.

- `model: string or null`

  The model that is evaluated, if applicable.

- `name: string or null`

  The name of the evaluation run.

- `object: "eval.run"`

  The type of the object. Always "eval.run".

  - `"eval.run"`

- `per_model_usage: array of object { cached_tokens, completion_tokens, invocation_count, 3 more }  or null`

  Usage statistics for each model during the evaluation run.

  - `cached_tokens: number`

    The number of tokens retrieved from cache.

  - `completion_tokens: number`

    The number of completion tokens generated.

  - `invocation_count: number`

    The number of invocations.

  - `model_name: string`

    The name of the model.

  - `prompt_tokens: number`

    The number of prompt tokens used.

  - `total_tokens: number`

    The total number of tokens used.

- `per_testing_criteria_results: array of object { failed, passed, testing_criteria }  or null`

  Results per testing criteria applied during the evaluation run.

  - `failed: number`

    Number of tests failed for this criteria.

  - `passed: number`

    Number of tests passed for this criteria.

  - `testing_criteria: string`

    A description of the testing criteria.

- `report_url: string`

  The URL to the rendered evaluation run report on the UI dashboard.

- `result_counts: object { errored, failed, passed, total }`

  Counters summarizing the outcomes of the evaluation run.

  - `errored: number`

    Number of output items that resulted in an error.

  - `failed: number`

    Number of output items that failed to pass the evaluation.

  - `passed: number`

    Number of output items that passed the evaluation.

  - `total: number`

    Total number of executed output items.

- `status: string`

  The status of the evaluation run.

### Example

```http
curl https://api.openai.com/v1/evals/$EVAL_ID/runs/$RUN_ID \
    -H "Authorization: Bearer $OPENAI_API_KEY"
```

#### Response

```json
{
  "id": "id",
  "created_at": 0,
  "data_source": {
    "source": {
      "content": [
        {
          "item": {
            "foo": "bar"
          },
          "sample": {
            "foo": "bar"
          }
        }
      ],
      "type": "file_content"
    },
    "type": "jsonl"
  },
  "error": {
    "code": "code",
    "message": "message"
  },
  "eval_id": "eval_id",
  "metadata": {
    "foo": "string"
  },
  "model": "model",
  "name": "name",
  "object": "eval.run",
  "per_model_usage": [
    {
      "cached_tokens": 0,
      "completion_tokens": 0,
      "invocation_count": 0,
      "model_name": "model_name",
      "prompt_tokens": 0,
      "total_tokens": 0
    }
  ],
  "per_testing_criteria_results": [
    {
      "failed": 0,
      "passed": 0,
      "testing_criteria": "testing_criteria"
    }
  ],
  "report_url": "https://example.com",
  "result_counts": {
    "errored": 0,
    "failed": 0,
    "passed": 0,
    "total": 0
  },
  "status": "status"
}
```

### Example

```http
curl https://api.openai.com/v1/evals/eval_67abd54d9b0081909a86353f6fb9317a/runs/evalrun_67abd54d60ec8190832b46859da808f7 \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json"
```

#### Response

```json
{
  "object": "eval.run",
  "id": "evalrun_67abd54d60ec8190832b46859da808f7",
  "eval_id": "eval_67abd54d9b0081909a86353f6fb9317a",
  "report_url": "https://platform.openai.com/evaluations/eval_67abd54d9b0081909a86353f6fb9317a?run_id=evalrun_67abd54d60ec8190832b46859da808f7",
  "status": "queued",
  "model": "gpt-6-astra",
  "name": "gpt-6-astra",
  "created_at": 1743092069,
  "result_counts": {
    "total": 0,
    "errored": 0,
    "failed": 0,
    "passed": 0
  },
  "per_model_usage": null,
  "per_testing_criteria_results": null,
  "data_source": {
    "type": "completions",
    "source": {
      "type": "file_content",
      "content": [
        {
          "item": {
            "input": "Tech Company Launches Advanced Artificial Intelligence Platform",
            "ground_truth": "Technology"
          }
        },
        {
          "item": {
            "input": "Central Bank Increases Interest Rates Amid Inflation Concerns",
            "ground_truth": "Markets"
          }
        },
        {
          "item": {
            "input": "International Summit Addresses Climate Change Strategies",
            "ground_truth": "World"
          }
        },
        {
          "item": {
            "input": "Major Retailer Reports Record-Breaking Holiday Sales",
            "ground_truth": "Business"
          }
        },
        {
          "item": {
            "input": "National Team Qualifies for World Championship Finals",
            "ground_truth": "Sports"
          }
        },
        {
          "item": {
            "input": "Stock Markets Rally After Positive Economic Data Released",
            "ground_truth": "Markets"
          }
        },
        {
          "item": {
            "input": "Global Manufacturer Announces Merger with Competitor",
            "ground_truth": "Business"
          }
        },
        {
          "item": {
            "input": "Breakthrough in Renewable Energy Technology Unveiled",
            "ground_truth": "Technology"
          }
        },
        {
          "item": {
            "input": "World Leaders Sign Historic Climate Agreement",
            "ground_truth": "World"
          }
        },
        {
          "item": {
            "input": "Professional Athlete Sets New Record in Championship Event",
            "ground_truth": "Sports"
          }
        },
        {
          "item": {
            "input": "Financial Institutions Adapt to New Regulatory Requirements",
            "ground_truth": "Business"
          }
        },
        {
          "item": {
            "input": "Tech Conference Showcases Advances in Artificial Intelligence",
            "ground_truth": "Technology"
          }
        },
        {
          "item": {
            "input": "Global Markets Respond to Oil Price Fluctuations",
            "ground_truth": "Markets"
          }
        },
        {
          "item": {
            "input": "International Cooperation Strengthened Through New Treaty",
            "ground_truth": "World"
          }
        },
        {
          "item": {
            "input": "Sports League Announces Revised Schedule for Upcoming Season",
            "ground_truth": "Sports"
          }
        }
      ]
    },
    "input_messages": {
      "type": "template",
      "template": [
        {
          "type": "message",
          "role": "developer",
          "content": {
            "type": "input_text",
            "text": "Categorize a given news headline into one of the following topics: Technology, Markets, World, Business, or Sports.\n\n# Steps\n\n1. Analyze the content of the news headline to understand its primary focus.\n2. Extract the subject matter, identifying any key indicators or keywords.\n3. Use the identified indicators to determine the most suitable category out of the five options: Technology, Markets, World, Business, or Sports.\n4. Ensure only one category is selected per headline.\n\n# Output Format\n\nRespond with the chosen category as a single word. For instance: \"Technology\", \"Markets\", \"World\", \"Business\", or \"Sports\".\n\n# Examples\n\n**Input**: \"Apple Unveils New iPhone Model, Featuring Advanced AI Features\"  \n**Output**: \"Technology\"\n\n**Input**: \"Global Stocks Mixed as Investors Await Central Bank Decisions\"  \n**Output**: \"Markets\"\n\n**Input**: \"War in Ukraine: Latest Updates on Negotiation Status\"  \n**Output**: \"World\"\n\n**Input**: \"Microsoft in Talks to Acquire Gaming Company for $2 Billion\"  \n**Output**: \"Business\"\n\n**Input**: \"Manchester United Secures Win in Premier League Football Match\"  \n**Output**: \"Sports\" \n\n# Notes\n\n- If the headline appears to fit into more than one category, choose the most dominant theme.\n- Keywords or phrases such as \"stocks\", \"company acquisition\", \"match\", or technological brands can be good indicators for classification.\n"
          }
        },
        {
          "type": "message",
          "role": "user",
          "content": {
            "type": "input_text",
            "text": "{{item.input}}"
          }
        }
      ]
    },
    "model": "gpt-6-astra",
    "sampling_params": {
      "max_completions_tokens": 2048
    }
  },
  "error": null,
  "metadata": {}
}
```

## Domain Types

### Create Eval Completions Run Data Source

- `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

  A CompletionsRunDataSource object describing a model sampling configuration.

  - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

    Determines what populates the `item` namespace in this run's data source.

    - `EvalJSONLFileContentSource object { content, type }`

      - `content: array of object { item, sample }`

        The content of the jsonl file.

        - `item: map[unknown]`

        - `sample: optional map[unknown]`

      - `type: "file_content"`

        The type of jsonl source. Always `file_content`.

        - `"file_content"`

    - `EvalJSONLFileIDSource object { id, type }`

      - `id: string`

        The identifier of the file.

      - `type: "file_id"`

        The type of jsonl source. Always `file_id`.

        - `"file_id"`

    - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

      A StoredCompletionsRunDataSource configuration describing a set of filters

      - `type: "stored_completions"`

        The type of source. Always `stored_completions`.

        - `"stored_completions"`

      - `created_after: optional number or null`

        An optional Unix timestamp to filter items created after this time.

      - `created_before: optional number or null`

        An optional Unix timestamp to filter items created before this time.

      - `limit: optional number or null`

        An optional maximum number of items to return.

      - `metadata: optional Metadata or null`

        Set of 16 key-value pairs that can be attached to an object. This can be
        useful for storing additional information about the object in a structured
        format, and querying for objects via API or the dashboard.

        Keys are strings with a maximum length of 64 characters. Values are strings
        with a maximum length of 512 characters.

      - `model: optional string or null`

        An optional model to filter by (e.g., 'gpt-6-astra').

  - `type: "completions"`

    The type of run data source. Always `completions`.

    - `"completions"`

  - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

    Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

    - `TemplateInputMessages object { template, type }`

      - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

        A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

        - `EasyInputMessage object { content, role, phase, type }`

          A message input to the model with a role indicating instruction following
          hierarchy. Instructions given with the `developer` or `system` role take
          precedence over instructions given with the `user` role. Messages with the
          `assistant` role are presumed to have been generated by the model in previous
          interactions.

          - `content: string or ResponseInputMessageContentList`

            Text, image, or audio input to the model, used to generate a response.
            Can also contain previous assistant responses.

            - `TextInput = string`

              A text input to the model.

            - `ResponseInputMessageContentList = array of ResponseInputContent`

              A list of one or many input items to the model, containing different content
              types.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

                - `text: string`

                  The text input to the model.

                - `type: "input_text"`

                  The type of the input item. Always `input_text`.

                  - `"input_text"`

                - `prompt_cache_breakpoint: optional object { mode }`

                  Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                  - `mode: "explicit"`

                    The breakpoint mode. Always `explicit`.

                    - `"explicit"`

              - `ResponseInputImage object { detail, type, file_id, 2 more }`

                An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                - `detail: ImageDetail`

                  The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                  - `"low"`

                  - `"high"`

                  - `"auto"`

                  - `"original"`

                - `type: "input_image"`

                  The type of the input item. Always `input_image`.

                  - `"input_image"`

                - `file_id: optional string or null`

                  The ID of the file to be sent to the model.

                - `image_url: optional string or null`

                  The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                - `prompt_cache_breakpoint: optional object { mode }`

                  Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                  - `mode: "explicit"`

                    The breakpoint mode. Always `explicit`.

                    - `"explicit"`

              - `ResponseInputFile object { type, detail, file_data, 4 more }`

                A file input to the model.

                - `type: "input_file"`

                  The type of the input item. Always `input_file`.

                  - `"input_file"`

                - `detail: optional "auto" or "low" or "high"`

                  The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                  - `"auto"`

                  - `"low"`

                  - `"high"`

                - `file_data: optional string`

                  The content of the file to be sent to the model.

                - `file_id: optional string or null`

                  The ID of the file to be sent to the model.

                - `file_url: optional string`

                  The URL of the file to be sent to the model.

                - `filename: optional string`

                  The name of the file to be sent to the model.

                - `prompt_cache_breakpoint: optional object { mode }`

                  Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                  - `mode: "explicit"`

                    The breakpoint mode. Always `explicit`.

                    - `"explicit"`

          - `role: "user" or "assistant" or "system" or "developer"`

            The role of the message input. One of `user`, `assistant`, `system`, or
            `developer`.

            - `"user"`

            - `"assistant"`

            - `"system"`

            - `"developer"`

          - `phase: optional "commentary" or "final_answer" or null`

            Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
            For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
            phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

            - `"commentary"`

            - `"final_answer"`

          - `type: optional "message"`

            The type of the message input. Always `message`.

            - `"message"`

        - `EvalMessageObject object { content, role, type }`

          A message input to the model with a role indicating instruction following
          hierarchy. Instructions given with the `developer` or `system` role take
          precedence over instructions given with the `user` role. Messages with the
          `assistant` role are presumed to have been generated by the model in previous
          interactions.

          - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

            Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

            - `TextInput = string`

              A text input to the model.

            - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

              A text input to the model.

            - `OutputText object { text, type }`

              A text output from the model.

              - `text: string`

                The text output from the model.

              - `type: "output_text"`

                The type of the output text. Always `output_text`.

                - `"output_text"`

            - `InputImage object { image_url, type, detail }`

              An image input block used within EvalItem content arrays.

              - `image_url: string`

                The URL of the image input.

              - `type: "input_image"`

                The type of the image input. Always `input_image`.

                - `"input_image"`

              - `detail: optional string`

                The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

            - `ResponseInputAudio object { input_audio, type }`

              An audio input to the model.

              - `input_audio: object { data, format }`

                - `data: string`

                  Base64-encoded audio data.

                - `format: "mp3" or "wav"`

                  The format of the audio data. Currently supported formats are `mp3` and
                  `wav`.

                  - `"mp3"`

                  - `"wav"`

              - `type: "input_audio"`

                The type of the input item. Always `input_audio`.

                - `"input_audio"`

            - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

              A list of inputs, each of which may be either an input text, output text, input
              image, or input audio object.

              - `TextInput = string`

                A text input to the model.

              - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                A text input to the model.

              - `OutputText object { text, type }`

                A text output from the model.

                - `text: string`

                  The text output from the model.

                - `type: "output_text"`

                  The type of the output text. Always `output_text`.

                  - `"output_text"`

              - `InputImage object { image_url, type, detail }`

                An image input block used within EvalItem content arrays.

                - `image_url: string`

                  The URL of the image input.

                - `type: "input_image"`

                  The type of the image input. Always `input_image`.

                  - `"input_image"`

                - `detail: optional string`

                  The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

              - `ResponseInputAudio object { input_audio, type }`

                An audio input to the model.

          - `role: "user" or "assistant" or "system" or "developer"`

            The role of the message input. One of `user`, `assistant`, `system`, or
            `developer`.

            - `"user"`

            - `"assistant"`

            - `"system"`

            - `"developer"`

          - `type: optional "message"`

            The type of the message input. Always `message`.

            - `"message"`

      - `type: "template"`

        The type of input messages. Always `template`.

        - `"template"`

    - `ItemReferenceInputMessages object { item_reference, type }`

      - `item_reference: string`

        A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

      - `type: "item_reference"`

        The type of input messages. Always `item_reference`.

        - `"item_reference"`

  - `model: optional string`

    The name of the model to use for generating completions (e.g. "o3-mini").

  - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

    - `max_completion_tokens: optional number`

      The maximum number of tokens in the generated output.

    - `reasoning_effort: optional ReasoningEffort or null`

      Constrains effort on reasoning for reasoning models. Currently supported
      values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
      Reducing reasoning effort can result in faster responses and fewer tokens
      used on reasoning in a response. Not all reasoning models support every
      value. See the
      [reasoning guide](/api/docs/guides/reasoning)
      for model-specific support.

      - `"none"`

      - `"minimal"`

      - `"low"`

      - `"medium"`

      - `"high"`

      - `"xhigh"`

      - `"max"`

    - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

      An object specifying the format that the model must output.

      Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
      Structured Outputs which ensures the model will match your supplied JSON
      schema. Learn more in the [Structured Outputs
      guide](/api/docs/guides/structured-outputs).

      Setting to `{ "type": "json_object" }` enables the older JSON mode, which
      ensures the message the model generates is valid JSON. Using `json_schema`
      is preferred for models that support it.

      - `ResponseFormatText object { type }`

        Default response format. Used to generate text responses.

        - `type: "text"`

          The type of response format being defined. Always `text`.

          - `"text"`

      - `ResponseFormatJSONSchema object { json_schema, type }`

        JSON Schema response format. Used to generate structured JSON responses.
        Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

        - `json_schema: object { name, description, schema, strict }`

          Structured Outputs configuration options, including a JSON Schema.

          - `name: string`

            The name of the response format. Must be a-z, A-Z, 0-9, or contain
            underscores and dashes, with a maximum length of 64.

          - `description: optional string`

            A description of what the response format is for, used by the model to
            determine how to respond in the format.

          - `schema: optional map[unknown]`

            The schema for the response format, described as a JSON Schema object.
            Learn how to build JSON schemas [here](https://json-schema.org/).

          - `strict: optional boolean or null`

            Whether to enable strict schema adherence when generating the output.
            If set to true, the model will always follow the exact schema defined
            in the `schema` field. Only a subset of JSON Schema is supported when
            `strict` is `true`. To learn more, read the [Structured Outputs
            guide](/api/docs/guides/structured-outputs).

        - `type: "json_schema"`

          The type of response format being defined. Always `json_schema`.

          - `"json_schema"`

      - `ResponseFormatJSONObject object { type }`

        JSON object response format. An older method of generating JSON responses.
        Using `json_schema` is recommended for models that support it. Note that the
        model will not generate JSON without a system or user message instructing it
        to do so.

        - `type: "json_object"`

          The type of response format being defined. Always `json_object`.

          - `"json_object"`

    - `seed: optional number`

      A seed value to initialize the randomness, during sampling.

    - `temperature: optional number`

      A higher temperature increases randomness in the outputs.

    - `tools: optional array of ChatCompletionFunctionTool`

      A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

      - `function: FunctionDefinition`

        - `name: string`

          The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

        - `description: optional string`

          A description of what the function does, used by the model to choose when and how to call the function.

        - `parameters: optional FunctionParameters`

          The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

          Omitting `parameters` defines a function with an empty parameter list.

        - `strict: optional boolean or null`

          Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

      - `type: "function"`

        The type of the tool. Currently, only `function` is supported.

        - `"function"`

    - `top_p: optional number`

      An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

### Create Eval JSONL Run Data Source

- `CreateEvalJSONLRunDataSource object { source, type }`

  A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

  - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

    Determines what populates the `item` namespace in the data source.

    - `EvalJSONLFileContentSource object { content, type }`

      - `content: array of object { item, sample }`

        The content of the jsonl file.

        - `item: map[unknown]`

        - `sample: optional map[unknown]`

      - `type: "file_content"`

        The type of jsonl source. Always `file_content`.

        - `"file_content"`

    - `EvalJSONLFileIDSource object { id, type }`

      - `id: string`

        The identifier of the file.

      - `type: "file_id"`

        The type of jsonl source. Always `file_id`.

        - `"file_id"`

  - `type: "jsonl"`

    The type of data source. Always `jsonl`.

    - `"jsonl"`

### Eval API Error

- `EvalAPIError object { code, message }`

  An object representing an error response from the Eval API.

  - `code: string`

    The error code.

  - `message: string`

    The error message.

### Run Cancel Response

- `RunCancelResponse object { id, created_at, data_source, 11 more }`

  A schema representing an evaluation run.

  - `id: string`

    Unique identifier for the evaluation run.

  - `created_at: number`

    Unix timestamp (in seconds) when the evaluation run was created.

  - `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }`

    Information about the run's data source.

    - `CreateEvalJSONLRunDataSource object { source, type }`

      A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

        Determines what populates the `item` namespace in the data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

      - `type: "jsonl"`

        The type of data source. Always `jsonl`.

        - `"jsonl"`

    - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

      A CompletionsRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

          A StoredCompletionsRunDataSource configuration describing a set of filters

          - `type: "stored_completions"`

            The type of source. Always `stored_completions`.

            - `"stored_completions"`

          - `created_after: optional number or null`

            An optional Unix timestamp to filter items created after this time.

          - `created_before: optional number or null`

            An optional Unix timestamp to filter items created before this time.

          - `limit: optional number or null`

            An optional maximum number of items to return.

          - `metadata: optional Metadata or null`

            Set of 16 key-value pairs that can be attached to an object. This can be
            useful for storing additional information about the object in a structured
            format, and querying for objects via API or the dashboard.

            Keys are strings with a maximum length of 64 characters. Values are strings
            with a maximum length of 512 characters.

          - `model: optional string or null`

            An optional model to filter by (e.g., 'gpt-6-astra').

      - `type: "completions"`

        The type of run data source. Always `completions`.

        - `"completions"`

      - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `TemplateInputMessages object { template, type }`

          - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `EasyInputMessage object { content, role, phase, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputMessageContentList`

                Text, image, or audio input to the model, used to generate a response.
                Can also contain previous assistant responses.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputMessageContentList = array of ResponseInputContent`

                  A list of one or many input items to the model, containing different content
                  types.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                    - `text: string`

                      The text input to the model.

                    - `type: "input_text"`

                      The type of the input item. Always `input_text`.

                      - `"input_text"`

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputImage object { detail, type, file_id, 2 more }`

                    An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                    - `detail: ImageDetail`

                      The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                      - `"low"`

                      - `"high"`

                      - `"auto"`

                      - `"original"`

                    - `type: "input_image"`

                      The type of the input item. Always `input_image`.

                      - `"input_image"`

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `image_url: optional string or null`

                      The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputFile object { type, detail, file_data, 4 more }`

                    A file input to the model.

                    - `type: "input_file"`

                      The type of the input item. Always `input_file`.

                      - `"input_file"`

                    - `detail: optional "auto" or "low" or "high"`

                      The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                      - `"auto"`

                      - `"low"`

                      - `"high"`

                    - `file_data: optional string`

                      The content of the file to be sent to the model.

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `file_url: optional string`

                      The URL of the file to be sent to the model.

                    - `filename: optional string`

                      The name of the file to be sent to the model.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `phase: optional "commentary" or "final_answer" or null`

                Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
                For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
                phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

                - `"commentary"`

                - `"final_answer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                  - `input_audio: object { data, format }`

                    - `data: string`

                      Base64-encoded audio data.

                    - `format: "mp3" or "wav"`

                      The format of the audio data. Currently supported formats are `mp3` and
                      `wav`.

                      - `"mp3"`

                      - `"wav"`

                  - `type: "input_audio"`

                    The type of the input item. Always `input_audio`.

                    - `"input_audio"`

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

                  - `TextInput = string`

                    A text input to the model.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                  - `OutputText object { text, type }`

                    A text output from the model.

                    - `text: string`

                      The text output from the model.

                    - `type: "output_text"`

                      The type of the output text. Always `output_text`.

                      - `"output_text"`

                  - `InputImage object { image_url, type, detail }`

                    An image input block used within EvalItem content arrays.

                    - `image_url: string`

                      The URL of the image input.

                    - `type: "input_image"`

                      The type of the image input. Always `input_image`.

                      - `"input_image"`

                    - `detail: optional string`

                      The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                  - `ResponseInputAudio object { input_audio, type }`

                    An audio input to the model.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `ItemReferenceInputMessages object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

          - `"none"`

          - `"minimal"`

          - `"low"`

          - `"medium"`

          - `"high"`

          - `"xhigh"`

          - `"max"`

        - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

          An object specifying the format that the model must output.

          Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
          Structured Outputs which ensures the model will match your supplied JSON
          schema. Learn more in the [Structured Outputs
          guide](/api/docs/guides/structured-outputs).

          Setting to `{ "type": "json_object" }` enables the older JSON mode, which
          ensures the message the model generates is valid JSON. Using `json_schema`
          is preferred for models that support it.

          - `ResponseFormatText object { type }`

            Default response format. Used to generate text responses.

            - `type: "text"`

              The type of response format being defined. Always `text`.

              - `"text"`

          - `ResponseFormatJSONSchema object { json_schema, type }`

            JSON Schema response format. Used to generate structured JSON responses.
            Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

            - `json_schema: object { name, description, schema, strict }`

              Structured Outputs configuration options, including a JSON Schema.

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `schema: optional map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `type: "json_schema"`

              The type of response format being defined. Always `json_schema`.

              - `"json_schema"`

          - `ResponseFormatJSONObject object { type }`

            JSON object response format. An older method of generating JSON responses.
            Using `json_schema` is recommended for models that support it. Note that the
            model will not generate JSON without a system or user message instructing it
            to do so.

            - `type: "json_object"`

              The type of response format being defined. Always `json_object`.

              - `"json_object"`

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `tools: optional array of ChatCompletionFunctionTool`

          A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

          - `function: FunctionDefinition`

            - `name: string`

              The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

            - `description: optional string`

              A description of what the function does, used by the model to choose when and how to call the function.

            - `parameters: optional FunctionParameters`

              The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

              Omitting `parameters` defines a function with an empty parameter list.

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

          - `type: "function"`

            The type of the tool. Currently, only `function` is supported.

            - `"function"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

    - `ResponsesRunDataSource object { source, type, input_messages, 2 more }`

      A ResponsesRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or EvalResponsesSource { type, created_after, created_before, 8 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `EvalResponsesSource object { type, created_after, created_before, 8 more }`

          A EvalResponsesSource object describing a run data source configuration.

          - `type: "responses"`

            The type of run data source. Always `responses`.

            - `"responses"`

          - `created_after: optional number or null`

            Only include items created after this timestamp (inclusive). This is a query parameter used to select responses.

          - `created_before: optional number or null`

            Only include items created before this timestamp (inclusive). This is a query parameter used to select responses.

          - `instructions_search: optional string or null`

            Optional string to search the 'instructions' field. This is a query parameter used to select responses.

          - `metadata: optional unknown or null`

            Metadata filter for the responses. This is a query parameter used to select responses.

          - `model: optional string or null`

            The name of the model to find responses for. This is a query parameter used to select responses.

          - `reasoning_effort: optional ReasoningEffort or null`

            Constrains effort on reasoning for reasoning models. Currently supported
            values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
            Reducing reasoning effort can result in faster responses and fewer tokens
            used on reasoning in a response. Not all reasoning models support every
            value. See the
            [reasoning guide](/api/docs/guides/reasoning)
            for model-specific support.

          - `temperature: optional number or null`

            Sampling temperature. This is a query parameter used to select responses.

          - `tools: optional array of string or null`

            List of tool names. This is a query parameter used to select responses.

          - `top_p: optional number or null`

            Nucleus sampling parameter. This is a query parameter used to select responses.

          - `users: optional array of string or null`

            List of user identifiers. This is a query parameter used to select responses.

      - `type: "responses"`

        The type of run data source. Always `responses`.

        - `"responses"`

      - `input_messages: optional InputMessagesTemplate { template, type }  or InputMessagesItemReference { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `InputMessagesTemplate object { template, type }`

          - `template: array of ChatMessage { content, role }  or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `ChatMessage object { content, role }`

              - `content: string`

                The content of the message.

              - `role: string`

                The role of the message (e.g. "system", "assistant", "user").

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `InputMessagesItemReference object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.name"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `text: optional object { format }`

          Configuration options for a text response from the model. Can be plain
          text or structured JSON data. Learn more:

          - [Text inputs and outputs](/api/docs/guides/text)
          - [Structured Outputs](/api/docs/guides/structured-outputs)

          - `format: optional ResponseFormatTextConfig`

            An object specifying the format that the model must output.

            Configuring `{ "type": "json_schema" }` enables Structured Outputs,
            which ensures the model will match your supplied JSON schema. Learn more in the
            [Structured Outputs guide](/api/docs/guides/structured-outputs).

            The default format is `{ "type": "text" }` with no additional options.

            **Not recommended for gpt-4o and newer models:**

            Setting to `{ "type": "json_object" }` enables the older JSON mode, which
            ensures the message the model generates is valid JSON. Using `json_schema`
            is preferred for models that support it.

            - `ResponseFormatText object { type }`

              Default response format. Used to generate text responses.

            - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }`

              JSON Schema response format. Used to generate structured JSON responses.
              Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `schema: map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `type: "json_schema"`

                The type of response format being defined. Always `json_schema`.

                - `"json_schema"`

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `ResponseFormatJSONObject object { type }`

              JSON object response format. An older method of generating JSON responses.
              Using `json_schema` is recommended for models that support it. Note that the
              model will not generate JSON without a system or user message instructing it
              to do so.

        - `tools: optional array of Function { name, parameters, strict, 6 more }  or FileSearch { type, vector_store_ids, filters, 2 more }  or Computer { type }  or 13 more`

          An array of tools the model may call while generating a response. You
          can specify which tool to use by setting the `tool_choice` parameter.

          The two categories of tools you can provide the model are:

          - **Built-in tools**: Tools that are provided by OpenAI that extend the
            model's capabilities, like [web search](/api/docs/guides/tools-web-search)
            or [file search](/api/docs/guides/tools-file-search). Learn more about
            [built-in tools](/api/docs/guides/tools).
          - **Function calls (custom tools)**: Functions that are defined by you,
            enabling the model to call your own code. Learn more about
            [function calling](/api/docs/guides/function-calling).

          - `Function object { name, parameters, strict, 6 more }`

            Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling).

            - `name: string`

              The name of the function to call.

            - `parameters: map[unknown] or null`

              A JSON schema object describing the parameters of the function.

            - `strict: boolean or null`

              Whether strict parameter validation is enforced for this function tool.

            - `type: "function"`

              The type of the function tool. Always `function`.

              - `"function"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

            - `defer_loading: optional boolean`

              Whether this function is deferred and loaded via tool search.

            - `description: optional string or null`

              A description of the function. Used by the model to determine whether or not to call the function.

            - `output_schema: optional map[unknown] or null`

              A JSON schema object describing the JSON value encoded in string outputs for this function.

          - `FileSearch object { type, vector_store_ids, filters, 2 more }`

            A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search).

            - `type: "file_search"`

              The type of the file search tool. Always `file_search`.

              - `"file_search"`

            - `vector_store_ids: array of string`

              The IDs of the vector stores to search.

            - `filters: optional ComparisonFilter or CompoundFilter or null`

              A filter to apply.

              - `ComparisonFilter object { key, type, value }`

                A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                - `key: string`

                  The key to compare against the value.

                - `type: "eq" or "ne" or "gt" or 5 more`

                  Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`.

                  - `eq`: equals
                  - `ne`: not equal
                  - `gt`: greater than
                  - `gte`: greater than or equal
                  - `lt`: less than
                  - `lte`: less than or equal
                  - `in`: in
                  - `nin`: not in

                  - `"eq"`

                  - `"ne"`

                  - `"gt"`

                  - `"gte"`

                  - `"lt"`

                  - `"lte"`

                  - `"in"`

                  - `"nin"`

                - `value: string or number or boolean or array of string or number`

                  The value to compare against the attribute key; supports string, number, or boolean types.

                  - `string`

                  - `number`

                  - `boolean`

                  - `array of string or number`

                    - `string`

                    - `number`

              - `CompoundFilter object { filters, type }`

                Combine multiple filters using `and` or `or`.

                - `filters: array of ComparisonFilter or CompoundFilter`

                  Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.

                  - `ComparisonFilter object { key, type, value }`

                    A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                  - `CompoundFilter object { filters, type }`

                    Combine multiple filters using `and` or `or`.

                - `type: "and" or "or"`

                  Type of operation: `and` or `or`.

                  - `"and"`

                  - `"or"`

            - `max_num_results: optional number`

              The maximum number of results to return. This number should be between 1 and 50 inclusive.

            - `ranking_options: optional object { hybrid_search, ranker, score_threshold }`

              Ranking options for search.

              - `hybrid_search: optional object { embedding_weight, text_weight }`

                Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

                - `embedding_weight: number`

                  The weight of the embedding in the reciprocal ranking fusion.

                - `text_weight: number`

                  The weight of the text in the reciprocal ranking fusion.

              - `ranker: optional "auto" or "default-2024-11-15"`

                The ranker to use for the file search.

                - `"auto"`

                - `"default-2024-11-15"`

              - `score_threshold: optional number`

                The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

          - `Computer object { type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `type: "computer"`

              The type of the computer tool. Always `computer`.

              - `"computer"`

          - `ComputerUsePreview object { display_height, display_width, environment, type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `display_height: number`

              The height of the computer display.

            - `display_width: number`

              The width of the computer display.

            - `environment: "windows" or "mac" or "linux" or 2 more`

              The type of computer environment to control.

              - `"windows"`

              - `"mac"`

              - `"linux"`

              - `"ubuntu"`

              - `"browser"`

            - `type: "computer_use_preview"`

              The type of the computer use tool. Always `computer_use_preview`.

              - `"computer_use_preview"`

          - `WebSearch object { type, external_web_access, filters, 2 more }`

            Search the Internet for sources related to the prompt. Learn more about the
            [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search" or "web_search_2025_08_26"`

              The type of the web search tool. One of `web_search` or `web_search_2025_08_26`.

              - `"web_search"`

              - `"web_search_2025_08_26"`

            - `external_web_access: optional boolean`

              Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

            - `filters: optional object { allowed_domains }  or null`

              Filters for the search.

              - `allowed_domains: optional array of string or null`

                Allowed domains for the search. If not provided, all domains are allowed.
                Subdomains of the provided domains are allowed as well.

                Example: `["pubmed.ncbi.nlm.nih.gov"]`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { city, country, region, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the
              United States. To avoid this fallback, pass `{"type": "approximate"}` without
              location fields. To localize results, provide the relevant location fields.

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

              - `type: optional "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

          - `Mcp object { server_label, type, allowed_callers, 9 more }`

            Give the model access to additional tools via remote Model Context Protocol
            (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp).

            - `server_label: string`

              A label for this MCP server, used to identify it in tool calls.

            - `type: "mcp"`

              The type of the MCP tool. Always `mcp`.

              - `"mcp"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names }  or null`

              List of allowed tool names or a filter object.

              - `McpAllowedTools = array of string`

                A string array of allowed tool names

              - `McpToolFilter object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

            - `authorization: optional string`

              An OAuth access token that can be used with a remote MCP server, either
              with a custom MCP server URL or a service connector. Your application
              must handle the OAuth authorization flow and provide the token here.

            - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more`

              Identifier for service connectors, like those available in ChatGPT. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more
              about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors).

              This field is deprecated for models released after September 1, 2026.
              Use `server_url` to connect to a remote MCP server, or `tunnel_id` to
              connect through a Secure MCP Tunnel.

              Currently supported `connector_id` values are:

              - Dropbox: `connector_dropbox`
              - Gmail: `connector_gmail`
              - Google Calendar: `connector_googlecalendar`
              - Google Drive: `connector_googledrive`
              - Microsoft Teams: `connector_microsoftteams`
              - Outlook Calendar: `connector_outlookcalendar`
              - Outlook Email: `connector_outlookemail`
              - SharePoint: `connector_sharepoint`

              - `"connector_dropbox"`

              - `"connector_gmail"`

              - `"connector_googlecalendar"`

              - `"connector_googledrive"`

              - `"connector_microsoftteams"`

              - `"connector_outlookcalendar"`

              - `"connector_outlookemail"`

              - `"connector_sharepoint"`

            - `defer_loading: optional boolean`

              Whether this MCP tool is deferred and discovered via tool search.

            - `headers: optional map[string] or null`

              Optional HTTP headers to send to the MCP server. Use for authentication
              or other purposes.

            - `require_approval: optional McpToolApprovalFilter { always, never }  or "always" or "never" or null`

              Specify which of the MCP server's tools require approval.

              - `McpToolApprovalFilter object { always, never }`

                Specify which of the MCP server's tools require approval. Can be
                `always`, `never`, or a filter object associated with tools
                that require approval.

                - `always: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

                - `never: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

              - `McpToolApprovalSetting = "always" or "never"`

                Specify a single approval policy for all tools. One of `always` or
                `never`. When set to `always`, all tools will require approval. When
                set to `never`, all tools will not require approval.

                - `"always"`

                - `"never"`

            - `server_description: optional string`

              Optional description of the MCP server, used to provide more context.

            - `server_url: optional string`

              The URL for the MCP server. One of `server_url`, `connector_id`, or
              `tunnel_id` must be provided.

            - `tunnel_id: optional string`

              The Secure MCP Tunnel ID to use instead of a direct server URL. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided.

          - `CodeInterpreter object { container, type, allowed_callers }`

            A tool that runs Python code to help generate a response to a prompt.

            - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }`

              The code interpreter container. Can be a container ID or an object that
              specifies uploaded file IDs to make available to your code, along with an
              optional `memory_limit` setting.

              - `string`

                The container ID.

              - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }`

                Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

                - `type: "auto"`

                  Always `auto`.

                  - `"auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the code interpreter container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                    - `type: "disabled"`

                      Disable outbound network access. Always `disabled`.

                      - `"disabled"`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                    - `allowed_domains: array of string`

                      A list of allowed domains when type is `allowlist`.

                    - `type: "allowlist"`

                      Allow outbound network access only to specified domains. Always `allowlist`.

                      - `"allowlist"`

                    - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret`

                      Optional domain-scoped secrets for allowlisted domains.

                      - `domain: string`

                        The domain associated with the secret.

                      - `name: string`

                        The name of the secret to inject for the domain.

                      - `value: string`

                        The secret value to inject for the domain.

            - `type: "code_interpreter"`

              The type of the code interpreter tool. Always `code_interpreter`.

              - `"code_interpreter"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

          - `ProgrammaticToolCalling object { type }`

            - `type: "programmatic_tool_calling"`

              The type of the tool. Always `programmatic_tool_calling`.

              - `"programmatic_tool_calling"`

          - `ImageGeneration object { type, action, background, 9 more }`

            A tool that generates images using the GPT image models.

            - `type: "image_generation"`

              The type of the image generation tool. Always `image_generation`.

              - `"image_generation"`

            - `action: optional "generate" or "edit" or "auto"`

              Whether to generate a new image or edit an existing image. Default: `auto`.

              - `"generate"`

              - `"edit"`

              - `"auto"`

            - `background: optional "transparent" or "opaque" or "auto"`

              Set the background of the generated image. One of `transparent`, `opaque`,
              or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including
              their `2026-09-08` snapshots, support `opaque` and `transparent`
              backgrounds. Transparent backgrounds are available for supported GPT Image
              models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in
              preview. When using `transparent`, set the output format to `png` or `webp`.
              Default: `auto`.

              - `"transparent"`

              - `"opaque"`

              - `"auto"`

            - `input_fidelity: optional "high" or "low" or null`

              Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models.

              - `"high"`

              - `"low"`

            - `input_image_mask: optional object { file_id, image_url }`

              Optional mask for inpainting. Contains `image_url`
              (string, optional) and `file_id` (string, optional).

              - `file_id: optional string`

                File ID for the mask image.

              - `image_url: optional string`

                Base64-encoded mask image.

            - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

              The image generation model to use. One of `gpt-image-1`,
              `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
              `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
              `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
              `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
              `gpt-image-1`.

              - `string`

              - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

                The image generation model to use. One of `gpt-image-1`,
                `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
                `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
                `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
                `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
                `gpt-image-1`.

                - `"gpt-image-1"`

                - `"gpt-image-1-mini"`

                - `"gpt-image-1.5"`

                - `"gpt-image-2"`

                - `"gpt-image-2-2026-04-21"`

                - `"gpt-image-2.5-sunburst"`

                - `"gpt-image-2.5-sunburst-2026-09-08"`

                - `"gpt-image-2.5-flare"`

                - `"gpt-image-2.5-flare-2026-09-08"`

            - `moderation: optional "auto" or "low"`

              Moderation level for the generated image. Default: `auto`.

              - `"auto"`

              - `"low"`

            - `output_compression: optional number`

              Compression level for the output image. Default: 100.

            - `output_format: optional "png" or "webp" or "jpeg"`

              The output format of the generated image. One of `png`, `webp`, or
              `jpeg`. Default: `png`.

              - `"png"`

              - `"webp"`

              - `"jpeg"`

            - `partial_images: optional number`

              Number of partial images to generate in streaming mode, from 0 (default value) to 3.

            - `quality: optional "low" or "medium" or "high" or 3 more`

              The quality of the generated image. The GPT image models support `low`,
              `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`,
              including their `2026-09-08` snapshots, also support `xhigh` and `max`.
              Default: `auto`.

              - `"low"`

              - `"medium"`

              - `"high"`

              - `"xhigh"`

              - `"max"`

              - `"auto"`

            - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"`

              The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

              - `string`

              - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"`

                The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

                - `"1024x1024"`

                - `"1024x1536"`

                - `"1536x1024"`

                - `"auto"`

          - `LocalShell object { type }`

            A tool that allows the model to execute shell commands in a local environment.

            - `type: "local_shell"`

              The type of the local shell tool. Always `local_shell`.

              - `"local_shell"`

          - `Shell object { type, allowed_callers, environment }`

            A tool that allows the model to execute shell commands.

            - `type: "shell"`

              The type of the shell tool. Always `shell`.

              - `"shell"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null`

              - `ContainerAuto object { type, file_ids, memory_limit, 2 more }`

                - `type: "container_auto"`

                  Automatically creates a container for this request

                  - `"container_auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                - `skills: optional array of SkillReference or InlineSkill`

                  An optional list of skills referenced by id or inline data.

                  - `SkillReference object { skill_id, type, version }`

                    - `skill_id: string`

                      The ID of the referenced skill.

                    - `type: "skill_reference"`

                      References a skill created with the /v1/skills endpoint.

                      - `"skill_reference"`

                    - `version: optional string`

                      Optional skill version. Use a positive integer or 'latest'. Omit for default.

                  - `InlineSkill object { description, name, source, type }`

                    - `description: string`

                      The description of the skill.

                    - `name: string`

                      The name of the skill.

                    - `source: InlineSkillSource`

                      Inline skill payload

                      - `data: string`

                        Base64-encoded skill zip bundle.

                      - `media_type: "application/zip"`

                        The media type of the inline skill payload. Must be `application/zip`.

                        - `"application/zip"`

                      - `type: "base64"`

                        The type of the inline skill source. Must be `base64`.

                        - `"base64"`

                    - `type: "inline"`

                      Defines an inline skill for this request.

                      - `"inline"`

              - `LocalEnvironment object { type, skills }`

                - `type: "local"`

                  Use a local computer environment.

                  - `"local"`

                - `skills: optional array of LocalSkill`

                  An optional list of skills.

                  - `description: string`

                    The description of the skill.

                  - `name: string`

                    The name of the skill.

                  - `path: string`

                    The path to the directory containing the skill.

              - `ContainerReference object { container_id, type }`

                - `container_id: string`

                  The ID of the referenced container.

                - `type: "container_reference"`

                  References a container created with the /v1/containers endpoint

                  - `"container_reference"`

          - `Custom object { name, type, allowed_callers, 4 more }`

            A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

            - `name: string`

              The name of the custom tool, used to identify it in tool calls.

            - `type: "custom"`

              The type of the custom tool. Always `custom`.

              - `"custom"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

              Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

            - `defer_loading: optional boolean`

              Whether this tool should be deferred and discovered via tool search.

            - `description: optional string`

              Optional description of the custom tool, used to provide more context.

            - `format: optional CustomToolInputFormat`

              The input format for the custom tool. Default is unconstrained text.

              - `Text object { type }`

                Unconstrained free-form text.

                - `type: "text"`

                  Unconstrained text format. Always `text`.

                  - `"text"`

              - `Grammar object { definition, syntax, type }`

                A grammar defined by the user.

                - `definition: string`

                  The grammar definition.

                - `syntax: "lark" or "regex"`

                  The syntax of the grammar definition. One of `lark` or `regex`.

                  - `"lark"`

                  - `"regex"`

                - `type: "grammar"`

                  Grammar format. Always `grammar`.

                  - `"grammar"`

          - `Namespace object { description, name, tools, type }`

            Groups function/custom tools under a shared namespace.

            - `description: string`

              A description of the namespace shown to the model.

            - `name: string`

              The namespace name used in tool calls (for example, `crm`).

            - `tools: array of Function { name, type, allowed_callers, 6 more }  or Custom { name, type, allowed_callers, 4 more }`

              The function/custom tools available inside this namespace.

              - `Function object { name, type, allowed_callers, 6 more }`

                - `name: string`

                - `type: "function"`

                  - `"function"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this function should be deferred and discovered via tool search.

                - `description: optional string or null`

                - `output_schema: optional map[unknown] or null`

                  A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

                - `parameters: optional unknown or null`

                - `strict: optional boolean or null`

                  Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

              - `Custom object { name, type, allowed_callers, 4 more }`

                A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

                - `name: string`

                  The name of the custom tool, used to identify it in tool calls.

                - `type: "custom"`

                  The type of the custom tool. Always `custom`.

                  - `"custom"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this tool should be deferred and discovered via tool search.

                - `description: optional string`

                  Optional description of the custom tool, used to provide more context.

                - `format: optional CustomToolInputFormat`

                  The input format for the custom tool. Default is unconstrained text.

            - `type: "namespace"`

              The type of the tool. Always `namespace`.

              - `"namespace"`

          - `ToolSearch object { type, description, execution, parameters }`

            Hosted or BYOT tool search configuration for deferred tools.

            - `type: "tool_search"`

              The type of the tool. Always `tool_search`.

              - `"tool_search"`

            - `description: optional string or null`

              Description shown to the model for a client-executed tool search tool.

            - `execution: optional "server" or "client"`

              Whether tool search is executed by the server or by the client.

              - `"server"`

              - `"client"`

            - `parameters: optional unknown or null`

              Parameter schema for a client-executed tool search tool.

          - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }`

            This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search_preview" or "web_search_preview_2025_03_11"`

              The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`.

              - `"web_search_preview"`

              - `"web_search_preview_2025_03_11"`

            - `search_content_types: optional array of "text" or "image"`

              - `"text"`

              - `"image"`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { type, city, country, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields.

              - `type: "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

          - `ApplyPatch object { type, allowed_callers }`

            Allows the assistant to create, delete, or update files using unified diffs.

            - `type: "apply_patch"`

              The type of the tool. Always `apply_patch`.

              - `"apply_patch"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

  - `error: EvalAPIError or null`

    An object representing an error response from the Eval API.

    - `code: string`

      The error code.

    - `message: string`

      The error message.

  - `eval_id: string`

    The identifier of the associated evaluation.

  - `metadata: Metadata or null`

    Set of 16 key-value pairs that can be attached to an object. This can be
    useful for storing additional information about the object in a structured
    format, and querying for objects via API or the dashboard.

    Keys are strings with a maximum length of 64 characters. Values are strings
    with a maximum length of 512 characters.

  - `model: string or null`

    The model that is evaluated, if applicable.

  - `name: string or null`

    The name of the evaluation run.

  - `object: "eval.run"`

    The type of the object. Always "eval.run".

    - `"eval.run"`

  - `per_model_usage: array of object { cached_tokens, completion_tokens, invocation_count, 3 more }  or null`

    Usage statistics for each model during the evaluation run.

    - `cached_tokens: number`

      The number of tokens retrieved from cache.

    - `completion_tokens: number`

      The number of completion tokens generated.

    - `invocation_count: number`

      The number of invocations.

    - `model_name: string`

      The name of the model.

    - `prompt_tokens: number`

      The number of prompt tokens used.

    - `total_tokens: number`

      The total number of tokens used.

  - `per_testing_criteria_results: array of object { failed, passed, testing_criteria }  or null`

    Results per testing criteria applied during the evaluation run.

    - `failed: number`

      Number of tests failed for this criteria.

    - `passed: number`

      Number of tests passed for this criteria.

    - `testing_criteria: string`

      A description of the testing criteria.

  - `report_url: string`

    The URL to the rendered evaluation run report on the UI dashboard.

  - `result_counts: object { errored, failed, passed, total }`

    Counters summarizing the outcomes of the evaluation run.

    - `errored: number`

      Number of output items that resulted in an error.

    - `failed: number`

      Number of output items that failed to pass the evaluation.

    - `passed: number`

      Number of output items that passed the evaluation.

    - `total: number`

      Total number of executed output items.

  - `status: string`

    The status of the evaluation run.

### Run Create Response

- `RunCreateResponse object { id, created_at, data_source, 11 more }`

  A schema representing an evaluation run.

  - `id: string`

    Unique identifier for the evaluation run.

  - `created_at: number`

    Unix timestamp (in seconds) when the evaluation run was created.

  - `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }`

    Information about the run's data source.

    - `CreateEvalJSONLRunDataSource object { source, type }`

      A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

        Determines what populates the `item` namespace in the data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

      - `type: "jsonl"`

        The type of data source. Always `jsonl`.

        - `"jsonl"`

    - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

      A CompletionsRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

          A StoredCompletionsRunDataSource configuration describing a set of filters

          - `type: "stored_completions"`

            The type of source. Always `stored_completions`.

            - `"stored_completions"`

          - `created_after: optional number or null`

            An optional Unix timestamp to filter items created after this time.

          - `created_before: optional number or null`

            An optional Unix timestamp to filter items created before this time.

          - `limit: optional number or null`

            An optional maximum number of items to return.

          - `metadata: optional Metadata or null`

            Set of 16 key-value pairs that can be attached to an object. This can be
            useful for storing additional information about the object in a structured
            format, and querying for objects via API or the dashboard.

            Keys are strings with a maximum length of 64 characters. Values are strings
            with a maximum length of 512 characters.

          - `model: optional string or null`

            An optional model to filter by (e.g., 'gpt-6-astra').

      - `type: "completions"`

        The type of run data source. Always `completions`.

        - `"completions"`

      - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `TemplateInputMessages object { template, type }`

          - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `EasyInputMessage object { content, role, phase, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputMessageContentList`

                Text, image, or audio input to the model, used to generate a response.
                Can also contain previous assistant responses.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputMessageContentList = array of ResponseInputContent`

                  A list of one or many input items to the model, containing different content
                  types.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                    - `text: string`

                      The text input to the model.

                    - `type: "input_text"`

                      The type of the input item. Always `input_text`.

                      - `"input_text"`

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputImage object { detail, type, file_id, 2 more }`

                    An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                    - `detail: ImageDetail`

                      The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                      - `"low"`

                      - `"high"`

                      - `"auto"`

                      - `"original"`

                    - `type: "input_image"`

                      The type of the input item. Always `input_image`.

                      - `"input_image"`

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `image_url: optional string or null`

                      The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputFile object { type, detail, file_data, 4 more }`

                    A file input to the model.

                    - `type: "input_file"`

                      The type of the input item. Always `input_file`.

                      - `"input_file"`

                    - `detail: optional "auto" or "low" or "high"`

                      The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                      - `"auto"`

                      - `"low"`

                      - `"high"`

                    - `file_data: optional string`

                      The content of the file to be sent to the model.

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `file_url: optional string`

                      The URL of the file to be sent to the model.

                    - `filename: optional string`

                      The name of the file to be sent to the model.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `phase: optional "commentary" or "final_answer" or null`

                Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
                For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
                phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

                - `"commentary"`

                - `"final_answer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                  - `input_audio: object { data, format }`

                    - `data: string`

                      Base64-encoded audio data.

                    - `format: "mp3" or "wav"`

                      The format of the audio data. Currently supported formats are `mp3` and
                      `wav`.

                      - `"mp3"`

                      - `"wav"`

                  - `type: "input_audio"`

                    The type of the input item. Always `input_audio`.

                    - `"input_audio"`

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

                  - `TextInput = string`

                    A text input to the model.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                  - `OutputText object { text, type }`

                    A text output from the model.

                    - `text: string`

                      The text output from the model.

                    - `type: "output_text"`

                      The type of the output text. Always `output_text`.

                      - `"output_text"`

                  - `InputImage object { image_url, type, detail }`

                    An image input block used within EvalItem content arrays.

                    - `image_url: string`

                      The URL of the image input.

                    - `type: "input_image"`

                      The type of the image input. Always `input_image`.

                      - `"input_image"`

                    - `detail: optional string`

                      The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                  - `ResponseInputAudio object { input_audio, type }`

                    An audio input to the model.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `ItemReferenceInputMessages object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

          - `"none"`

          - `"minimal"`

          - `"low"`

          - `"medium"`

          - `"high"`

          - `"xhigh"`

          - `"max"`

        - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

          An object specifying the format that the model must output.

          Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
          Structured Outputs which ensures the model will match your supplied JSON
          schema. Learn more in the [Structured Outputs
          guide](/api/docs/guides/structured-outputs).

          Setting to `{ "type": "json_object" }` enables the older JSON mode, which
          ensures the message the model generates is valid JSON. Using `json_schema`
          is preferred for models that support it.

          - `ResponseFormatText object { type }`

            Default response format. Used to generate text responses.

            - `type: "text"`

              The type of response format being defined. Always `text`.

              - `"text"`

          - `ResponseFormatJSONSchema object { json_schema, type }`

            JSON Schema response format. Used to generate structured JSON responses.
            Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

            - `json_schema: object { name, description, schema, strict }`

              Structured Outputs configuration options, including a JSON Schema.

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `schema: optional map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `type: "json_schema"`

              The type of response format being defined. Always `json_schema`.

              - `"json_schema"`

          - `ResponseFormatJSONObject object { type }`

            JSON object response format. An older method of generating JSON responses.
            Using `json_schema` is recommended for models that support it. Note that the
            model will not generate JSON without a system or user message instructing it
            to do so.

            - `type: "json_object"`

              The type of response format being defined. Always `json_object`.

              - `"json_object"`

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `tools: optional array of ChatCompletionFunctionTool`

          A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

          - `function: FunctionDefinition`

            - `name: string`

              The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

            - `description: optional string`

              A description of what the function does, used by the model to choose when and how to call the function.

            - `parameters: optional FunctionParameters`

              The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

              Omitting `parameters` defines a function with an empty parameter list.

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

          - `type: "function"`

            The type of the tool. Currently, only `function` is supported.

            - `"function"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

    - `ResponsesRunDataSource object { source, type, input_messages, 2 more }`

      A ResponsesRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or EvalResponsesSource { type, created_after, created_before, 8 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `EvalResponsesSource object { type, created_after, created_before, 8 more }`

          A EvalResponsesSource object describing a run data source configuration.

          - `type: "responses"`

            The type of run data source. Always `responses`.

            - `"responses"`

          - `created_after: optional number or null`

            Only include items created after this timestamp (inclusive). This is a query parameter used to select responses.

          - `created_before: optional number or null`

            Only include items created before this timestamp (inclusive). This is a query parameter used to select responses.

          - `instructions_search: optional string or null`

            Optional string to search the 'instructions' field. This is a query parameter used to select responses.

          - `metadata: optional unknown or null`

            Metadata filter for the responses. This is a query parameter used to select responses.

          - `model: optional string or null`

            The name of the model to find responses for. This is a query parameter used to select responses.

          - `reasoning_effort: optional ReasoningEffort or null`

            Constrains effort on reasoning for reasoning models. Currently supported
            values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
            Reducing reasoning effort can result in faster responses and fewer tokens
            used on reasoning in a response. Not all reasoning models support every
            value. See the
            [reasoning guide](/api/docs/guides/reasoning)
            for model-specific support.

          - `temperature: optional number or null`

            Sampling temperature. This is a query parameter used to select responses.

          - `tools: optional array of string or null`

            List of tool names. This is a query parameter used to select responses.

          - `top_p: optional number or null`

            Nucleus sampling parameter. This is a query parameter used to select responses.

          - `users: optional array of string or null`

            List of user identifiers. This is a query parameter used to select responses.

      - `type: "responses"`

        The type of run data source. Always `responses`.

        - `"responses"`

      - `input_messages: optional InputMessagesTemplate { template, type }  or InputMessagesItemReference { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `InputMessagesTemplate object { template, type }`

          - `template: array of ChatMessage { content, role }  or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `ChatMessage object { content, role }`

              - `content: string`

                The content of the message.

              - `role: string`

                The role of the message (e.g. "system", "assistant", "user").

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `InputMessagesItemReference object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.name"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `text: optional object { format }`

          Configuration options for a text response from the model. Can be plain
          text or structured JSON data. Learn more:

          - [Text inputs and outputs](/api/docs/guides/text)
          - [Structured Outputs](/api/docs/guides/structured-outputs)

          - `format: optional ResponseFormatTextConfig`

            An object specifying the format that the model must output.

            Configuring `{ "type": "json_schema" }` enables Structured Outputs,
            which ensures the model will match your supplied JSON schema. Learn more in the
            [Structured Outputs guide](/api/docs/guides/structured-outputs).

            The default format is `{ "type": "text" }` with no additional options.

            **Not recommended for gpt-4o and newer models:**

            Setting to `{ "type": "json_object" }` enables the older JSON mode, which
            ensures the message the model generates is valid JSON. Using `json_schema`
            is preferred for models that support it.

            - `ResponseFormatText object { type }`

              Default response format. Used to generate text responses.

            - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }`

              JSON Schema response format. Used to generate structured JSON responses.
              Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `schema: map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `type: "json_schema"`

                The type of response format being defined. Always `json_schema`.

                - `"json_schema"`

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `ResponseFormatJSONObject object { type }`

              JSON object response format. An older method of generating JSON responses.
              Using `json_schema` is recommended for models that support it. Note that the
              model will not generate JSON without a system or user message instructing it
              to do so.

        - `tools: optional array of Function { name, parameters, strict, 6 more }  or FileSearch { type, vector_store_ids, filters, 2 more }  or Computer { type }  or 13 more`

          An array of tools the model may call while generating a response. You
          can specify which tool to use by setting the `tool_choice` parameter.

          The two categories of tools you can provide the model are:

          - **Built-in tools**: Tools that are provided by OpenAI that extend the
            model's capabilities, like [web search](/api/docs/guides/tools-web-search)
            or [file search](/api/docs/guides/tools-file-search). Learn more about
            [built-in tools](/api/docs/guides/tools).
          - **Function calls (custom tools)**: Functions that are defined by you,
            enabling the model to call your own code. Learn more about
            [function calling](/api/docs/guides/function-calling).

          - `Function object { name, parameters, strict, 6 more }`

            Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling).

            - `name: string`

              The name of the function to call.

            - `parameters: map[unknown] or null`

              A JSON schema object describing the parameters of the function.

            - `strict: boolean or null`

              Whether strict parameter validation is enforced for this function tool.

            - `type: "function"`

              The type of the function tool. Always `function`.

              - `"function"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

            - `defer_loading: optional boolean`

              Whether this function is deferred and loaded via tool search.

            - `description: optional string or null`

              A description of the function. Used by the model to determine whether or not to call the function.

            - `output_schema: optional map[unknown] or null`

              A JSON schema object describing the JSON value encoded in string outputs for this function.

          - `FileSearch object { type, vector_store_ids, filters, 2 more }`

            A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search).

            - `type: "file_search"`

              The type of the file search tool. Always `file_search`.

              - `"file_search"`

            - `vector_store_ids: array of string`

              The IDs of the vector stores to search.

            - `filters: optional ComparisonFilter or CompoundFilter or null`

              A filter to apply.

              - `ComparisonFilter object { key, type, value }`

                A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                - `key: string`

                  The key to compare against the value.

                - `type: "eq" or "ne" or "gt" or 5 more`

                  Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`.

                  - `eq`: equals
                  - `ne`: not equal
                  - `gt`: greater than
                  - `gte`: greater than or equal
                  - `lt`: less than
                  - `lte`: less than or equal
                  - `in`: in
                  - `nin`: not in

                  - `"eq"`

                  - `"ne"`

                  - `"gt"`

                  - `"gte"`

                  - `"lt"`

                  - `"lte"`

                  - `"in"`

                  - `"nin"`

                - `value: string or number or boolean or array of string or number`

                  The value to compare against the attribute key; supports string, number, or boolean types.

                  - `string`

                  - `number`

                  - `boolean`

                  - `array of string or number`

                    - `string`

                    - `number`

              - `CompoundFilter object { filters, type }`

                Combine multiple filters using `and` or `or`.

                - `filters: array of ComparisonFilter or CompoundFilter`

                  Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.

                  - `ComparisonFilter object { key, type, value }`

                    A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                  - `CompoundFilter object { filters, type }`

                    Combine multiple filters using `and` or `or`.

                - `type: "and" or "or"`

                  Type of operation: `and` or `or`.

                  - `"and"`

                  - `"or"`

            - `max_num_results: optional number`

              The maximum number of results to return. This number should be between 1 and 50 inclusive.

            - `ranking_options: optional object { hybrid_search, ranker, score_threshold }`

              Ranking options for search.

              - `hybrid_search: optional object { embedding_weight, text_weight }`

                Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

                - `embedding_weight: number`

                  The weight of the embedding in the reciprocal ranking fusion.

                - `text_weight: number`

                  The weight of the text in the reciprocal ranking fusion.

              - `ranker: optional "auto" or "default-2024-11-15"`

                The ranker to use for the file search.

                - `"auto"`

                - `"default-2024-11-15"`

              - `score_threshold: optional number`

                The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

          - `Computer object { type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `type: "computer"`

              The type of the computer tool. Always `computer`.

              - `"computer"`

          - `ComputerUsePreview object { display_height, display_width, environment, type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `display_height: number`

              The height of the computer display.

            - `display_width: number`

              The width of the computer display.

            - `environment: "windows" or "mac" or "linux" or 2 more`

              The type of computer environment to control.

              - `"windows"`

              - `"mac"`

              - `"linux"`

              - `"ubuntu"`

              - `"browser"`

            - `type: "computer_use_preview"`

              The type of the computer use tool. Always `computer_use_preview`.

              - `"computer_use_preview"`

          - `WebSearch object { type, external_web_access, filters, 2 more }`

            Search the Internet for sources related to the prompt. Learn more about the
            [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search" or "web_search_2025_08_26"`

              The type of the web search tool. One of `web_search` or `web_search_2025_08_26`.

              - `"web_search"`

              - `"web_search_2025_08_26"`

            - `external_web_access: optional boolean`

              Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

            - `filters: optional object { allowed_domains }  or null`

              Filters for the search.

              - `allowed_domains: optional array of string or null`

                Allowed domains for the search. If not provided, all domains are allowed.
                Subdomains of the provided domains are allowed as well.

                Example: `["pubmed.ncbi.nlm.nih.gov"]`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { city, country, region, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the
              United States. To avoid this fallback, pass `{"type": "approximate"}` without
              location fields. To localize results, provide the relevant location fields.

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

              - `type: optional "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

          - `Mcp object { server_label, type, allowed_callers, 9 more }`

            Give the model access to additional tools via remote Model Context Protocol
            (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp).

            - `server_label: string`

              A label for this MCP server, used to identify it in tool calls.

            - `type: "mcp"`

              The type of the MCP tool. Always `mcp`.

              - `"mcp"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names }  or null`

              List of allowed tool names or a filter object.

              - `McpAllowedTools = array of string`

                A string array of allowed tool names

              - `McpToolFilter object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

            - `authorization: optional string`

              An OAuth access token that can be used with a remote MCP server, either
              with a custom MCP server URL or a service connector. Your application
              must handle the OAuth authorization flow and provide the token here.

            - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more`

              Identifier for service connectors, like those available in ChatGPT. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more
              about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors).

              This field is deprecated for models released after September 1, 2026.
              Use `server_url` to connect to a remote MCP server, or `tunnel_id` to
              connect through a Secure MCP Tunnel.

              Currently supported `connector_id` values are:

              - Dropbox: `connector_dropbox`
              - Gmail: `connector_gmail`
              - Google Calendar: `connector_googlecalendar`
              - Google Drive: `connector_googledrive`
              - Microsoft Teams: `connector_microsoftteams`
              - Outlook Calendar: `connector_outlookcalendar`
              - Outlook Email: `connector_outlookemail`
              - SharePoint: `connector_sharepoint`

              - `"connector_dropbox"`

              - `"connector_gmail"`

              - `"connector_googlecalendar"`

              - `"connector_googledrive"`

              - `"connector_microsoftteams"`

              - `"connector_outlookcalendar"`

              - `"connector_outlookemail"`

              - `"connector_sharepoint"`

            - `defer_loading: optional boolean`

              Whether this MCP tool is deferred and discovered via tool search.

            - `headers: optional map[string] or null`

              Optional HTTP headers to send to the MCP server. Use for authentication
              or other purposes.

            - `require_approval: optional McpToolApprovalFilter { always, never }  or "always" or "never" or null`

              Specify which of the MCP server's tools require approval.

              - `McpToolApprovalFilter object { always, never }`

                Specify which of the MCP server's tools require approval. Can be
                `always`, `never`, or a filter object associated with tools
                that require approval.

                - `always: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

                - `never: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

              - `McpToolApprovalSetting = "always" or "never"`

                Specify a single approval policy for all tools. One of `always` or
                `never`. When set to `always`, all tools will require approval. When
                set to `never`, all tools will not require approval.

                - `"always"`

                - `"never"`

            - `server_description: optional string`

              Optional description of the MCP server, used to provide more context.

            - `server_url: optional string`

              The URL for the MCP server. One of `server_url`, `connector_id`, or
              `tunnel_id` must be provided.

            - `tunnel_id: optional string`

              The Secure MCP Tunnel ID to use instead of a direct server URL. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided.

          - `CodeInterpreter object { container, type, allowed_callers }`

            A tool that runs Python code to help generate a response to a prompt.

            - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }`

              The code interpreter container. Can be a container ID or an object that
              specifies uploaded file IDs to make available to your code, along with an
              optional `memory_limit` setting.

              - `string`

                The container ID.

              - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }`

                Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

                - `type: "auto"`

                  Always `auto`.

                  - `"auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the code interpreter container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                    - `type: "disabled"`

                      Disable outbound network access. Always `disabled`.

                      - `"disabled"`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                    - `allowed_domains: array of string`

                      A list of allowed domains when type is `allowlist`.

                    - `type: "allowlist"`

                      Allow outbound network access only to specified domains. Always `allowlist`.

                      - `"allowlist"`

                    - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret`

                      Optional domain-scoped secrets for allowlisted domains.

                      - `domain: string`

                        The domain associated with the secret.

                      - `name: string`

                        The name of the secret to inject for the domain.

                      - `value: string`

                        The secret value to inject for the domain.

            - `type: "code_interpreter"`

              The type of the code interpreter tool. Always `code_interpreter`.

              - `"code_interpreter"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

          - `ProgrammaticToolCalling object { type }`

            - `type: "programmatic_tool_calling"`

              The type of the tool. Always `programmatic_tool_calling`.

              - `"programmatic_tool_calling"`

          - `ImageGeneration object { type, action, background, 9 more }`

            A tool that generates images using the GPT image models.

            - `type: "image_generation"`

              The type of the image generation tool. Always `image_generation`.

              - `"image_generation"`

            - `action: optional "generate" or "edit" or "auto"`

              Whether to generate a new image or edit an existing image. Default: `auto`.

              - `"generate"`

              - `"edit"`

              - `"auto"`

            - `background: optional "transparent" or "opaque" or "auto"`

              Set the background of the generated image. One of `transparent`, `opaque`,
              or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including
              their `2026-09-08` snapshots, support `opaque` and `transparent`
              backgrounds. Transparent backgrounds are available for supported GPT Image
              models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in
              preview. When using `transparent`, set the output format to `png` or `webp`.
              Default: `auto`.

              - `"transparent"`

              - `"opaque"`

              - `"auto"`

            - `input_fidelity: optional "high" or "low" or null`

              Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models.

              - `"high"`

              - `"low"`

            - `input_image_mask: optional object { file_id, image_url }`

              Optional mask for inpainting. Contains `image_url`
              (string, optional) and `file_id` (string, optional).

              - `file_id: optional string`

                File ID for the mask image.

              - `image_url: optional string`

                Base64-encoded mask image.

            - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

              The image generation model to use. One of `gpt-image-1`,
              `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
              `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
              `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
              `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
              `gpt-image-1`.

              - `string`

              - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

                The image generation model to use. One of `gpt-image-1`,
                `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
                `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
                `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
                `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
                `gpt-image-1`.

                - `"gpt-image-1"`

                - `"gpt-image-1-mini"`

                - `"gpt-image-1.5"`

                - `"gpt-image-2"`

                - `"gpt-image-2-2026-04-21"`

                - `"gpt-image-2.5-sunburst"`

                - `"gpt-image-2.5-sunburst-2026-09-08"`

                - `"gpt-image-2.5-flare"`

                - `"gpt-image-2.5-flare-2026-09-08"`

            - `moderation: optional "auto" or "low"`

              Moderation level for the generated image. Default: `auto`.

              - `"auto"`

              - `"low"`

            - `output_compression: optional number`

              Compression level for the output image. Default: 100.

            - `output_format: optional "png" or "webp" or "jpeg"`

              The output format of the generated image. One of `png`, `webp`, or
              `jpeg`. Default: `png`.

              - `"png"`

              - `"webp"`

              - `"jpeg"`

            - `partial_images: optional number`

              Number of partial images to generate in streaming mode, from 0 (default value) to 3.

            - `quality: optional "low" or "medium" or "high" or 3 more`

              The quality of the generated image. The GPT image models support `low`,
              `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`,
              including their `2026-09-08` snapshots, also support `xhigh` and `max`.
              Default: `auto`.

              - `"low"`

              - `"medium"`

              - `"high"`

              - `"xhigh"`

              - `"max"`

              - `"auto"`

            - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"`

              The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

              - `string`

              - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"`

                The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

                - `"1024x1024"`

                - `"1024x1536"`

                - `"1536x1024"`

                - `"auto"`

          - `LocalShell object { type }`

            A tool that allows the model to execute shell commands in a local environment.

            - `type: "local_shell"`

              The type of the local shell tool. Always `local_shell`.

              - `"local_shell"`

          - `Shell object { type, allowed_callers, environment }`

            A tool that allows the model to execute shell commands.

            - `type: "shell"`

              The type of the shell tool. Always `shell`.

              - `"shell"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null`

              - `ContainerAuto object { type, file_ids, memory_limit, 2 more }`

                - `type: "container_auto"`

                  Automatically creates a container for this request

                  - `"container_auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                - `skills: optional array of SkillReference or InlineSkill`

                  An optional list of skills referenced by id or inline data.

                  - `SkillReference object { skill_id, type, version }`

                    - `skill_id: string`

                      The ID of the referenced skill.

                    - `type: "skill_reference"`

                      References a skill created with the /v1/skills endpoint.

                      - `"skill_reference"`

                    - `version: optional string`

                      Optional skill version. Use a positive integer or 'latest'. Omit for default.

                  - `InlineSkill object { description, name, source, type }`

                    - `description: string`

                      The description of the skill.

                    - `name: string`

                      The name of the skill.

                    - `source: InlineSkillSource`

                      Inline skill payload

                      - `data: string`

                        Base64-encoded skill zip bundle.

                      - `media_type: "application/zip"`

                        The media type of the inline skill payload. Must be `application/zip`.

                        - `"application/zip"`

                      - `type: "base64"`

                        The type of the inline skill source. Must be `base64`.

                        - `"base64"`

                    - `type: "inline"`

                      Defines an inline skill for this request.

                      - `"inline"`

              - `LocalEnvironment object { type, skills }`

                - `type: "local"`

                  Use a local computer environment.

                  - `"local"`

                - `skills: optional array of LocalSkill`

                  An optional list of skills.

                  - `description: string`

                    The description of the skill.

                  - `name: string`

                    The name of the skill.

                  - `path: string`

                    The path to the directory containing the skill.

              - `ContainerReference object { container_id, type }`

                - `container_id: string`

                  The ID of the referenced container.

                - `type: "container_reference"`

                  References a container created with the /v1/containers endpoint

                  - `"container_reference"`

          - `Custom object { name, type, allowed_callers, 4 more }`

            A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

            - `name: string`

              The name of the custom tool, used to identify it in tool calls.

            - `type: "custom"`

              The type of the custom tool. Always `custom`.

              - `"custom"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

              Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

            - `defer_loading: optional boolean`

              Whether this tool should be deferred and discovered via tool search.

            - `description: optional string`

              Optional description of the custom tool, used to provide more context.

            - `format: optional CustomToolInputFormat`

              The input format for the custom tool. Default is unconstrained text.

              - `Text object { type }`

                Unconstrained free-form text.

                - `type: "text"`

                  Unconstrained text format. Always `text`.

                  - `"text"`

              - `Grammar object { definition, syntax, type }`

                A grammar defined by the user.

                - `definition: string`

                  The grammar definition.

                - `syntax: "lark" or "regex"`

                  The syntax of the grammar definition. One of `lark` or `regex`.

                  - `"lark"`

                  - `"regex"`

                - `type: "grammar"`

                  Grammar format. Always `grammar`.

                  - `"grammar"`

          - `Namespace object { description, name, tools, type }`

            Groups function/custom tools under a shared namespace.

            - `description: string`

              A description of the namespace shown to the model.

            - `name: string`

              The namespace name used in tool calls (for example, `crm`).

            - `tools: array of Function { name, type, allowed_callers, 6 more }  or Custom { name, type, allowed_callers, 4 more }`

              The function/custom tools available inside this namespace.

              - `Function object { name, type, allowed_callers, 6 more }`

                - `name: string`

                - `type: "function"`

                  - `"function"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this function should be deferred and discovered via tool search.

                - `description: optional string or null`

                - `output_schema: optional map[unknown] or null`

                  A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

                - `parameters: optional unknown or null`

                - `strict: optional boolean or null`

                  Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

              - `Custom object { name, type, allowed_callers, 4 more }`

                A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

                - `name: string`

                  The name of the custom tool, used to identify it in tool calls.

                - `type: "custom"`

                  The type of the custom tool. Always `custom`.

                  - `"custom"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this tool should be deferred and discovered via tool search.

                - `description: optional string`

                  Optional description of the custom tool, used to provide more context.

                - `format: optional CustomToolInputFormat`

                  The input format for the custom tool. Default is unconstrained text.

            - `type: "namespace"`

              The type of the tool. Always `namespace`.

              - `"namespace"`

          - `ToolSearch object { type, description, execution, parameters }`

            Hosted or BYOT tool search configuration for deferred tools.

            - `type: "tool_search"`

              The type of the tool. Always `tool_search`.

              - `"tool_search"`

            - `description: optional string or null`

              Description shown to the model for a client-executed tool search tool.

            - `execution: optional "server" or "client"`

              Whether tool search is executed by the server or by the client.

              - `"server"`

              - `"client"`

            - `parameters: optional unknown or null`

              Parameter schema for a client-executed tool search tool.

          - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }`

            This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search_preview" or "web_search_preview_2025_03_11"`

              The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`.

              - `"web_search_preview"`

              - `"web_search_preview_2025_03_11"`

            - `search_content_types: optional array of "text" or "image"`

              - `"text"`

              - `"image"`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { type, city, country, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields.

              - `type: "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

          - `ApplyPatch object { type, allowed_callers }`

            Allows the assistant to create, delete, or update files using unified diffs.

            - `type: "apply_patch"`

              The type of the tool. Always `apply_patch`.

              - `"apply_patch"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

  - `error: EvalAPIError or null`

    An object representing an error response from the Eval API.

    - `code: string`

      The error code.

    - `message: string`

      The error message.

  - `eval_id: string`

    The identifier of the associated evaluation.

  - `metadata: Metadata or null`

    Set of 16 key-value pairs that can be attached to an object. This can be
    useful for storing additional information about the object in a structured
    format, and querying for objects via API or the dashboard.

    Keys are strings with a maximum length of 64 characters. Values are strings
    with a maximum length of 512 characters.

  - `model: string or null`

    The model that is evaluated, if applicable.

  - `name: string or null`

    The name of the evaluation run.

  - `object: "eval.run"`

    The type of the object. Always "eval.run".

    - `"eval.run"`

  - `per_model_usage: array of object { cached_tokens, completion_tokens, invocation_count, 3 more }  or null`

    Usage statistics for each model during the evaluation run.

    - `cached_tokens: number`

      The number of tokens retrieved from cache.

    - `completion_tokens: number`

      The number of completion tokens generated.

    - `invocation_count: number`

      The number of invocations.

    - `model_name: string`

      The name of the model.

    - `prompt_tokens: number`

      The number of prompt tokens used.

    - `total_tokens: number`

      The total number of tokens used.

  - `per_testing_criteria_results: array of object { failed, passed, testing_criteria }  or null`

    Results per testing criteria applied during the evaluation run.

    - `failed: number`

      Number of tests failed for this criteria.

    - `passed: number`

      Number of tests passed for this criteria.

    - `testing_criteria: string`

      A description of the testing criteria.

  - `report_url: string`

    The URL to the rendered evaluation run report on the UI dashboard.

  - `result_counts: object { errored, failed, passed, total }`

    Counters summarizing the outcomes of the evaluation run.

    - `errored: number`

      Number of output items that resulted in an error.

    - `failed: number`

      Number of output items that failed to pass the evaluation.

    - `passed: number`

      Number of output items that passed the evaluation.

    - `total: number`

      Total number of executed output items.

  - `status: string`

    The status of the evaluation run.

### Run Delete Response

- `RunDeleteResponse object { deleted, object, run_id }`

  - `deleted: optional boolean`

  - `object: optional string`

  - `run_id: optional string`

### Run List Response

- `RunListResponse object { id, created_at, data_source, 11 more }`

  A schema representing an evaluation run.

  - `id: string`

    Unique identifier for the evaluation run.

  - `created_at: number`

    Unix timestamp (in seconds) when the evaluation run was created.

  - `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }`

    Information about the run's data source.

    - `CreateEvalJSONLRunDataSource object { source, type }`

      A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

        Determines what populates the `item` namespace in the data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

      - `type: "jsonl"`

        The type of data source. Always `jsonl`.

        - `"jsonl"`

    - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

      A CompletionsRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

          A StoredCompletionsRunDataSource configuration describing a set of filters

          - `type: "stored_completions"`

            The type of source. Always `stored_completions`.

            - `"stored_completions"`

          - `created_after: optional number or null`

            An optional Unix timestamp to filter items created after this time.

          - `created_before: optional number or null`

            An optional Unix timestamp to filter items created before this time.

          - `limit: optional number or null`

            An optional maximum number of items to return.

          - `metadata: optional Metadata or null`

            Set of 16 key-value pairs that can be attached to an object. This can be
            useful for storing additional information about the object in a structured
            format, and querying for objects via API or the dashboard.

            Keys are strings with a maximum length of 64 characters. Values are strings
            with a maximum length of 512 characters.

          - `model: optional string or null`

            An optional model to filter by (e.g., 'gpt-6-astra').

      - `type: "completions"`

        The type of run data source. Always `completions`.

        - `"completions"`

      - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `TemplateInputMessages object { template, type }`

          - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `EasyInputMessage object { content, role, phase, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputMessageContentList`

                Text, image, or audio input to the model, used to generate a response.
                Can also contain previous assistant responses.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputMessageContentList = array of ResponseInputContent`

                  A list of one or many input items to the model, containing different content
                  types.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                    - `text: string`

                      The text input to the model.

                    - `type: "input_text"`

                      The type of the input item. Always `input_text`.

                      - `"input_text"`

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputImage object { detail, type, file_id, 2 more }`

                    An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                    - `detail: ImageDetail`

                      The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                      - `"low"`

                      - `"high"`

                      - `"auto"`

                      - `"original"`

                    - `type: "input_image"`

                      The type of the input item. Always `input_image`.

                      - `"input_image"`

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `image_url: optional string or null`

                      The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputFile object { type, detail, file_data, 4 more }`

                    A file input to the model.

                    - `type: "input_file"`

                      The type of the input item. Always `input_file`.

                      - `"input_file"`

                    - `detail: optional "auto" or "low" or "high"`

                      The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                      - `"auto"`

                      - `"low"`

                      - `"high"`

                    - `file_data: optional string`

                      The content of the file to be sent to the model.

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `file_url: optional string`

                      The URL of the file to be sent to the model.

                    - `filename: optional string`

                      The name of the file to be sent to the model.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `phase: optional "commentary" or "final_answer" or null`

                Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
                For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
                phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

                - `"commentary"`

                - `"final_answer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                  - `input_audio: object { data, format }`

                    - `data: string`

                      Base64-encoded audio data.

                    - `format: "mp3" or "wav"`

                      The format of the audio data. Currently supported formats are `mp3` and
                      `wav`.

                      - `"mp3"`

                      - `"wav"`

                  - `type: "input_audio"`

                    The type of the input item. Always `input_audio`.

                    - `"input_audio"`

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

                  - `TextInput = string`

                    A text input to the model.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                  - `OutputText object { text, type }`

                    A text output from the model.

                    - `text: string`

                      The text output from the model.

                    - `type: "output_text"`

                      The type of the output text. Always `output_text`.

                      - `"output_text"`

                  - `InputImage object { image_url, type, detail }`

                    An image input block used within EvalItem content arrays.

                    - `image_url: string`

                      The URL of the image input.

                    - `type: "input_image"`

                      The type of the image input. Always `input_image`.

                      - `"input_image"`

                    - `detail: optional string`

                      The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                  - `ResponseInputAudio object { input_audio, type }`

                    An audio input to the model.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `ItemReferenceInputMessages object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

          - `"none"`

          - `"minimal"`

          - `"low"`

          - `"medium"`

          - `"high"`

          - `"xhigh"`

          - `"max"`

        - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

          An object specifying the format that the model must output.

          Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
          Structured Outputs which ensures the model will match your supplied JSON
          schema. Learn more in the [Structured Outputs
          guide](/api/docs/guides/structured-outputs).

          Setting to `{ "type": "json_object" }` enables the older JSON mode, which
          ensures the message the model generates is valid JSON. Using `json_schema`
          is preferred for models that support it.

          - `ResponseFormatText object { type }`

            Default response format. Used to generate text responses.

            - `type: "text"`

              The type of response format being defined. Always `text`.

              - `"text"`

          - `ResponseFormatJSONSchema object { json_schema, type }`

            JSON Schema response format. Used to generate structured JSON responses.
            Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

            - `json_schema: object { name, description, schema, strict }`

              Structured Outputs configuration options, including a JSON Schema.

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `schema: optional map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `type: "json_schema"`

              The type of response format being defined. Always `json_schema`.

              - `"json_schema"`

          - `ResponseFormatJSONObject object { type }`

            JSON object response format. An older method of generating JSON responses.
            Using `json_schema` is recommended for models that support it. Note that the
            model will not generate JSON without a system or user message instructing it
            to do so.

            - `type: "json_object"`

              The type of response format being defined. Always `json_object`.

              - `"json_object"`

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `tools: optional array of ChatCompletionFunctionTool`

          A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

          - `function: FunctionDefinition`

            - `name: string`

              The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

            - `description: optional string`

              A description of what the function does, used by the model to choose when and how to call the function.

            - `parameters: optional FunctionParameters`

              The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

              Omitting `parameters` defines a function with an empty parameter list.

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

          - `type: "function"`

            The type of the tool. Currently, only `function` is supported.

            - `"function"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

    - `ResponsesRunDataSource object { source, type, input_messages, 2 more }`

      A ResponsesRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or EvalResponsesSource { type, created_after, created_before, 8 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `EvalResponsesSource object { type, created_after, created_before, 8 more }`

          A EvalResponsesSource object describing a run data source configuration.

          - `type: "responses"`

            The type of run data source. Always `responses`.

            - `"responses"`

          - `created_after: optional number or null`

            Only include items created after this timestamp (inclusive). This is a query parameter used to select responses.

          - `created_before: optional number or null`

            Only include items created before this timestamp (inclusive). This is a query parameter used to select responses.

          - `instructions_search: optional string or null`

            Optional string to search the 'instructions' field. This is a query parameter used to select responses.

          - `metadata: optional unknown or null`

            Metadata filter for the responses. This is a query parameter used to select responses.

          - `model: optional string or null`

            The name of the model to find responses for. This is a query parameter used to select responses.

          - `reasoning_effort: optional ReasoningEffort or null`

            Constrains effort on reasoning for reasoning models. Currently supported
            values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
            Reducing reasoning effort can result in faster responses and fewer tokens
            used on reasoning in a response. Not all reasoning models support every
            value. See the
            [reasoning guide](/api/docs/guides/reasoning)
            for model-specific support.

          - `temperature: optional number or null`

            Sampling temperature. This is a query parameter used to select responses.

          - `tools: optional array of string or null`

            List of tool names. This is a query parameter used to select responses.

          - `top_p: optional number or null`

            Nucleus sampling parameter. This is a query parameter used to select responses.

          - `users: optional array of string or null`

            List of user identifiers. This is a query parameter used to select responses.

      - `type: "responses"`

        The type of run data source. Always `responses`.

        - `"responses"`

      - `input_messages: optional InputMessagesTemplate { template, type }  or InputMessagesItemReference { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `InputMessagesTemplate object { template, type }`

          - `template: array of ChatMessage { content, role }  or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `ChatMessage object { content, role }`

              - `content: string`

                The content of the message.

              - `role: string`

                The role of the message (e.g. "system", "assistant", "user").

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `InputMessagesItemReference object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.name"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `text: optional object { format }`

          Configuration options for a text response from the model. Can be plain
          text or structured JSON data. Learn more:

          - [Text inputs and outputs](/api/docs/guides/text)
          - [Structured Outputs](/api/docs/guides/structured-outputs)

          - `format: optional ResponseFormatTextConfig`

            An object specifying the format that the model must output.

            Configuring `{ "type": "json_schema" }` enables Structured Outputs,
            which ensures the model will match your supplied JSON schema. Learn more in the
            [Structured Outputs guide](/api/docs/guides/structured-outputs).

            The default format is `{ "type": "text" }` with no additional options.

            **Not recommended for gpt-4o and newer models:**

            Setting to `{ "type": "json_object" }` enables the older JSON mode, which
            ensures the message the model generates is valid JSON. Using `json_schema`
            is preferred for models that support it.

            - `ResponseFormatText object { type }`

              Default response format. Used to generate text responses.

            - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }`

              JSON Schema response format. Used to generate structured JSON responses.
              Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `schema: map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `type: "json_schema"`

                The type of response format being defined. Always `json_schema`.

                - `"json_schema"`

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `ResponseFormatJSONObject object { type }`

              JSON object response format. An older method of generating JSON responses.
              Using `json_schema` is recommended for models that support it. Note that the
              model will not generate JSON without a system or user message instructing it
              to do so.

        - `tools: optional array of Function { name, parameters, strict, 6 more }  or FileSearch { type, vector_store_ids, filters, 2 more }  or Computer { type }  or 13 more`

          An array of tools the model may call while generating a response. You
          can specify which tool to use by setting the `tool_choice` parameter.

          The two categories of tools you can provide the model are:

          - **Built-in tools**: Tools that are provided by OpenAI that extend the
            model's capabilities, like [web search](/api/docs/guides/tools-web-search)
            or [file search](/api/docs/guides/tools-file-search). Learn more about
            [built-in tools](/api/docs/guides/tools).
          - **Function calls (custom tools)**: Functions that are defined by you,
            enabling the model to call your own code. Learn more about
            [function calling](/api/docs/guides/function-calling).

          - `Function object { name, parameters, strict, 6 more }`

            Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling).

            - `name: string`

              The name of the function to call.

            - `parameters: map[unknown] or null`

              A JSON schema object describing the parameters of the function.

            - `strict: boolean or null`

              Whether strict parameter validation is enforced for this function tool.

            - `type: "function"`

              The type of the function tool. Always `function`.

              - `"function"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

            - `defer_loading: optional boolean`

              Whether this function is deferred and loaded via tool search.

            - `description: optional string or null`

              A description of the function. Used by the model to determine whether or not to call the function.

            - `output_schema: optional map[unknown] or null`

              A JSON schema object describing the JSON value encoded in string outputs for this function.

          - `FileSearch object { type, vector_store_ids, filters, 2 more }`

            A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search).

            - `type: "file_search"`

              The type of the file search tool. Always `file_search`.

              - `"file_search"`

            - `vector_store_ids: array of string`

              The IDs of the vector stores to search.

            - `filters: optional ComparisonFilter or CompoundFilter or null`

              A filter to apply.

              - `ComparisonFilter object { key, type, value }`

                A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                - `key: string`

                  The key to compare against the value.

                - `type: "eq" or "ne" or "gt" or 5 more`

                  Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`.

                  - `eq`: equals
                  - `ne`: not equal
                  - `gt`: greater than
                  - `gte`: greater than or equal
                  - `lt`: less than
                  - `lte`: less than or equal
                  - `in`: in
                  - `nin`: not in

                  - `"eq"`

                  - `"ne"`

                  - `"gt"`

                  - `"gte"`

                  - `"lt"`

                  - `"lte"`

                  - `"in"`

                  - `"nin"`

                - `value: string or number or boolean or array of string or number`

                  The value to compare against the attribute key; supports string, number, or boolean types.

                  - `string`

                  - `number`

                  - `boolean`

                  - `array of string or number`

                    - `string`

                    - `number`

              - `CompoundFilter object { filters, type }`

                Combine multiple filters using `and` or `or`.

                - `filters: array of ComparisonFilter or CompoundFilter`

                  Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.

                  - `ComparisonFilter object { key, type, value }`

                    A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                  - `CompoundFilter object { filters, type }`

                    Combine multiple filters using `and` or `or`.

                - `type: "and" or "or"`

                  Type of operation: `and` or `or`.

                  - `"and"`

                  - `"or"`

            - `max_num_results: optional number`

              The maximum number of results to return. This number should be between 1 and 50 inclusive.

            - `ranking_options: optional object { hybrid_search, ranker, score_threshold }`

              Ranking options for search.

              - `hybrid_search: optional object { embedding_weight, text_weight }`

                Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

                - `embedding_weight: number`

                  The weight of the embedding in the reciprocal ranking fusion.

                - `text_weight: number`

                  The weight of the text in the reciprocal ranking fusion.

              - `ranker: optional "auto" or "default-2024-11-15"`

                The ranker to use for the file search.

                - `"auto"`

                - `"default-2024-11-15"`

              - `score_threshold: optional number`

                The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

          - `Computer object { type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `type: "computer"`

              The type of the computer tool. Always `computer`.

              - `"computer"`

          - `ComputerUsePreview object { display_height, display_width, environment, type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `display_height: number`

              The height of the computer display.

            - `display_width: number`

              The width of the computer display.

            - `environment: "windows" or "mac" or "linux" or 2 more`

              The type of computer environment to control.

              - `"windows"`

              - `"mac"`

              - `"linux"`

              - `"ubuntu"`

              - `"browser"`

            - `type: "computer_use_preview"`

              The type of the computer use tool. Always `computer_use_preview`.

              - `"computer_use_preview"`

          - `WebSearch object { type, external_web_access, filters, 2 more }`

            Search the Internet for sources related to the prompt. Learn more about the
            [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search" or "web_search_2025_08_26"`

              The type of the web search tool. One of `web_search` or `web_search_2025_08_26`.

              - `"web_search"`

              - `"web_search_2025_08_26"`

            - `external_web_access: optional boolean`

              Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

            - `filters: optional object { allowed_domains }  or null`

              Filters for the search.

              - `allowed_domains: optional array of string or null`

                Allowed domains for the search. If not provided, all domains are allowed.
                Subdomains of the provided domains are allowed as well.

                Example: `["pubmed.ncbi.nlm.nih.gov"]`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { city, country, region, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the
              United States. To avoid this fallback, pass `{"type": "approximate"}` without
              location fields. To localize results, provide the relevant location fields.

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

              - `type: optional "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

          - `Mcp object { server_label, type, allowed_callers, 9 more }`

            Give the model access to additional tools via remote Model Context Protocol
            (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp).

            - `server_label: string`

              A label for this MCP server, used to identify it in tool calls.

            - `type: "mcp"`

              The type of the MCP tool. Always `mcp`.

              - `"mcp"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names }  or null`

              List of allowed tool names or a filter object.

              - `McpAllowedTools = array of string`

                A string array of allowed tool names

              - `McpToolFilter object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

            - `authorization: optional string`

              An OAuth access token that can be used with a remote MCP server, either
              with a custom MCP server URL or a service connector. Your application
              must handle the OAuth authorization flow and provide the token here.

            - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more`

              Identifier for service connectors, like those available in ChatGPT. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more
              about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors).

              This field is deprecated for models released after September 1, 2026.
              Use `server_url` to connect to a remote MCP server, or `tunnel_id` to
              connect through a Secure MCP Tunnel.

              Currently supported `connector_id` values are:

              - Dropbox: `connector_dropbox`
              - Gmail: `connector_gmail`
              - Google Calendar: `connector_googlecalendar`
              - Google Drive: `connector_googledrive`
              - Microsoft Teams: `connector_microsoftteams`
              - Outlook Calendar: `connector_outlookcalendar`
              - Outlook Email: `connector_outlookemail`
              - SharePoint: `connector_sharepoint`

              - `"connector_dropbox"`

              - `"connector_gmail"`

              - `"connector_googlecalendar"`

              - `"connector_googledrive"`

              - `"connector_microsoftteams"`

              - `"connector_outlookcalendar"`

              - `"connector_outlookemail"`

              - `"connector_sharepoint"`

            - `defer_loading: optional boolean`

              Whether this MCP tool is deferred and discovered via tool search.

            - `headers: optional map[string] or null`

              Optional HTTP headers to send to the MCP server. Use for authentication
              or other purposes.

            - `require_approval: optional McpToolApprovalFilter { always, never }  or "always" or "never" or null`

              Specify which of the MCP server's tools require approval.

              - `McpToolApprovalFilter object { always, never }`

                Specify which of the MCP server's tools require approval. Can be
                `always`, `never`, or a filter object associated with tools
                that require approval.

                - `always: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

                - `never: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

              - `McpToolApprovalSetting = "always" or "never"`

                Specify a single approval policy for all tools. One of `always` or
                `never`. When set to `always`, all tools will require approval. When
                set to `never`, all tools will not require approval.

                - `"always"`

                - `"never"`

            - `server_description: optional string`

              Optional description of the MCP server, used to provide more context.

            - `server_url: optional string`

              The URL for the MCP server. One of `server_url`, `connector_id`, or
              `tunnel_id` must be provided.

            - `tunnel_id: optional string`

              The Secure MCP Tunnel ID to use instead of a direct server URL. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided.

          - `CodeInterpreter object { container, type, allowed_callers }`

            A tool that runs Python code to help generate a response to a prompt.

            - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }`

              The code interpreter container. Can be a container ID or an object that
              specifies uploaded file IDs to make available to your code, along with an
              optional `memory_limit` setting.

              - `string`

                The container ID.

              - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }`

                Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

                - `type: "auto"`

                  Always `auto`.

                  - `"auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the code interpreter container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                    - `type: "disabled"`

                      Disable outbound network access. Always `disabled`.

                      - `"disabled"`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                    - `allowed_domains: array of string`

                      A list of allowed domains when type is `allowlist`.

                    - `type: "allowlist"`

                      Allow outbound network access only to specified domains. Always `allowlist`.

                      - `"allowlist"`

                    - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret`

                      Optional domain-scoped secrets for allowlisted domains.

                      - `domain: string`

                        The domain associated with the secret.

                      - `name: string`

                        The name of the secret to inject for the domain.

                      - `value: string`

                        The secret value to inject for the domain.

            - `type: "code_interpreter"`

              The type of the code interpreter tool. Always `code_interpreter`.

              - `"code_interpreter"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

          - `ProgrammaticToolCalling object { type }`

            - `type: "programmatic_tool_calling"`

              The type of the tool. Always `programmatic_tool_calling`.

              - `"programmatic_tool_calling"`

          - `ImageGeneration object { type, action, background, 9 more }`

            A tool that generates images using the GPT image models.

            - `type: "image_generation"`

              The type of the image generation tool. Always `image_generation`.

              - `"image_generation"`

            - `action: optional "generate" or "edit" or "auto"`

              Whether to generate a new image or edit an existing image. Default: `auto`.

              - `"generate"`

              - `"edit"`

              - `"auto"`

            - `background: optional "transparent" or "opaque" or "auto"`

              Set the background of the generated image. One of `transparent`, `opaque`,
              or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including
              their `2026-09-08` snapshots, support `opaque` and `transparent`
              backgrounds. Transparent backgrounds are available for supported GPT Image
              models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in
              preview. When using `transparent`, set the output format to `png` or `webp`.
              Default: `auto`.

              - `"transparent"`

              - `"opaque"`

              - `"auto"`

            - `input_fidelity: optional "high" or "low" or null`

              Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models.

              - `"high"`

              - `"low"`

            - `input_image_mask: optional object { file_id, image_url }`

              Optional mask for inpainting. Contains `image_url`
              (string, optional) and `file_id` (string, optional).

              - `file_id: optional string`

                File ID for the mask image.

              - `image_url: optional string`

                Base64-encoded mask image.

            - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

              The image generation model to use. One of `gpt-image-1`,
              `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
              `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
              `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
              `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
              `gpt-image-1`.

              - `string`

              - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

                The image generation model to use. One of `gpt-image-1`,
                `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
                `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
                `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
                `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
                `gpt-image-1`.

                - `"gpt-image-1"`

                - `"gpt-image-1-mini"`

                - `"gpt-image-1.5"`

                - `"gpt-image-2"`

                - `"gpt-image-2-2026-04-21"`

                - `"gpt-image-2.5-sunburst"`

                - `"gpt-image-2.5-sunburst-2026-09-08"`

                - `"gpt-image-2.5-flare"`

                - `"gpt-image-2.5-flare-2026-09-08"`

            - `moderation: optional "auto" or "low"`

              Moderation level for the generated image. Default: `auto`.

              - `"auto"`

              - `"low"`

            - `output_compression: optional number`

              Compression level for the output image. Default: 100.

            - `output_format: optional "png" or "webp" or "jpeg"`

              The output format of the generated image. One of `png`, `webp`, or
              `jpeg`. Default: `png`.

              - `"png"`

              - `"webp"`

              - `"jpeg"`

            - `partial_images: optional number`

              Number of partial images to generate in streaming mode, from 0 (default value) to 3.

            - `quality: optional "low" or "medium" or "high" or 3 more`

              The quality of the generated image. The GPT image models support `low`,
              `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`,
              including their `2026-09-08` snapshots, also support `xhigh` and `max`.
              Default: `auto`.

              - `"low"`

              - `"medium"`

              - `"high"`

              - `"xhigh"`

              - `"max"`

              - `"auto"`

            - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"`

              The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

              - `string`

              - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"`

                The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

                - `"1024x1024"`

                - `"1024x1536"`

                - `"1536x1024"`

                - `"auto"`

          - `LocalShell object { type }`

            A tool that allows the model to execute shell commands in a local environment.

            - `type: "local_shell"`

              The type of the local shell tool. Always `local_shell`.

              - `"local_shell"`

          - `Shell object { type, allowed_callers, environment }`

            A tool that allows the model to execute shell commands.

            - `type: "shell"`

              The type of the shell tool. Always `shell`.

              - `"shell"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null`

              - `ContainerAuto object { type, file_ids, memory_limit, 2 more }`

                - `type: "container_auto"`

                  Automatically creates a container for this request

                  - `"container_auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                - `skills: optional array of SkillReference or InlineSkill`

                  An optional list of skills referenced by id or inline data.

                  - `SkillReference object { skill_id, type, version }`

                    - `skill_id: string`

                      The ID of the referenced skill.

                    - `type: "skill_reference"`

                      References a skill created with the /v1/skills endpoint.

                      - `"skill_reference"`

                    - `version: optional string`

                      Optional skill version. Use a positive integer or 'latest'. Omit for default.

                  - `InlineSkill object { description, name, source, type }`

                    - `description: string`

                      The description of the skill.

                    - `name: string`

                      The name of the skill.

                    - `source: InlineSkillSource`

                      Inline skill payload

                      - `data: string`

                        Base64-encoded skill zip bundle.

                      - `media_type: "application/zip"`

                        The media type of the inline skill payload. Must be `application/zip`.

                        - `"application/zip"`

                      - `type: "base64"`

                        The type of the inline skill source. Must be `base64`.

                        - `"base64"`

                    - `type: "inline"`

                      Defines an inline skill for this request.

                      - `"inline"`

              - `LocalEnvironment object { type, skills }`

                - `type: "local"`

                  Use a local computer environment.

                  - `"local"`

                - `skills: optional array of LocalSkill`

                  An optional list of skills.

                  - `description: string`

                    The description of the skill.

                  - `name: string`

                    The name of the skill.

                  - `path: string`

                    The path to the directory containing the skill.

              - `ContainerReference object { container_id, type }`

                - `container_id: string`

                  The ID of the referenced container.

                - `type: "container_reference"`

                  References a container created with the /v1/containers endpoint

                  - `"container_reference"`

          - `Custom object { name, type, allowed_callers, 4 more }`

            A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

            - `name: string`

              The name of the custom tool, used to identify it in tool calls.

            - `type: "custom"`

              The type of the custom tool. Always `custom`.

              - `"custom"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

              Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

            - `defer_loading: optional boolean`

              Whether this tool should be deferred and discovered via tool search.

            - `description: optional string`

              Optional description of the custom tool, used to provide more context.

            - `format: optional CustomToolInputFormat`

              The input format for the custom tool. Default is unconstrained text.

              - `Text object { type }`

                Unconstrained free-form text.

                - `type: "text"`

                  Unconstrained text format. Always `text`.

                  - `"text"`

              - `Grammar object { definition, syntax, type }`

                A grammar defined by the user.

                - `definition: string`

                  The grammar definition.

                - `syntax: "lark" or "regex"`

                  The syntax of the grammar definition. One of `lark` or `regex`.

                  - `"lark"`

                  - `"regex"`

                - `type: "grammar"`

                  Grammar format. Always `grammar`.

                  - `"grammar"`

          - `Namespace object { description, name, tools, type }`

            Groups function/custom tools under a shared namespace.

            - `description: string`

              A description of the namespace shown to the model.

            - `name: string`

              The namespace name used in tool calls (for example, `crm`).

            - `tools: array of Function { name, type, allowed_callers, 6 more }  or Custom { name, type, allowed_callers, 4 more }`

              The function/custom tools available inside this namespace.

              - `Function object { name, type, allowed_callers, 6 more }`

                - `name: string`

                - `type: "function"`

                  - `"function"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this function should be deferred and discovered via tool search.

                - `description: optional string or null`

                - `output_schema: optional map[unknown] or null`

                  A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

                - `parameters: optional unknown or null`

                - `strict: optional boolean or null`

                  Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

              - `Custom object { name, type, allowed_callers, 4 more }`

                A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

                - `name: string`

                  The name of the custom tool, used to identify it in tool calls.

                - `type: "custom"`

                  The type of the custom tool. Always `custom`.

                  - `"custom"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this tool should be deferred and discovered via tool search.

                - `description: optional string`

                  Optional description of the custom tool, used to provide more context.

                - `format: optional CustomToolInputFormat`

                  The input format for the custom tool. Default is unconstrained text.

            - `type: "namespace"`

              The type of the tool. Always `namespace`.

              - `"namespace"`

          - `ToolSearch object { type, description, execution, parameters }`

            Hosted or BYOT tool search configuration for deferred tools.

            - `type: "tool_search"`

              The type of the tool. Always `tool_search`.

              - `"tool_search"`

            - `description: optional string or null`

              Description shown to the model for a client-executed tool search tool.

            - `execution: optional "server" or "client"`

              Whether tool search is executed by the server or by the client.

              - `"server"`

              - `"client"`

            - `parameters: optional unknown or null`

              Parameter schema for a client-executed tool search tool.

          - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }`

            This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search_preview" or "web_search_preview_2025_03_11"`

              The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`.

              - `"web_search_preview"`

              - `"web_search_preview_2025_03_11"`

            - `search_content_types: optional array of "text" or "image"`

              - `"text"`

              - `"image"`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { type, city, country, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields.

              - `type: "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

          - `ApplyPatch object { type, allowed_callers }`

            Allows the assistant to create, delete, or update files using unified diffs.

            - `type: "apply_patch"`

              The type of the tool. Always `apply_patch`.

              - `"apply_patch"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

  - `error: EvalAPIError or null`

    An object representing an error response from the Eval API.

    - `code: string`

      The error code.

    - `message: string`

      The error message.

  - `eval_id: string`

    The identifier of the associated evaluation.

  - `metadata: Metadata or null`

    Set of 16 key-value pairs that can be attached to an object. This can be
    useful for storing additional information about the object in a structured
    format, and querying for objects via API or the dashboard.

    Keys are strings with a maximum length of 64 characters. Values are strings
    with a maximum length of 512 characters.

  - `model: string or null`

    The model that is evaluated, if applicable.

  - `name: string or null`

    The name of the evaluation run.

  - `object: "eval.run"`

    The type of the object. Always "eval.run".

    - `"eval.run"`

  - `per_model_usage: array of object { cached_tokens, completion_tokens, invocation_count, 3 more }  or null`

    Usage statistics for each model during the evaluation run.

    - `cached_tokens: number`

      The number of tokens retrieved from cache.

    - `completion_tokens: number`

      The number of completion tokens generated.

    - `invocation_count: number`

      The number of invocations.

    - `model_name: string`

      The name of the model.

    - `prompt_tokens: number`

      The number of prompt tokens used.

    - `total_tokens: number`

      The total number of tokens used.

  - `per_testing_criteria_results: array of object { failed, passed, testing_criteria }  or null`

    Results per testing criteria applied during the evaluation run.

    - `failed: number`

      Number of tests failed for this criteria.

    - `passed: number`

      Number of tests passed for this criteria.

    - `testing_criteria: string`

      A description of the testing criteria.

  - `report_url: string`

    The URL to the rendered evaluation run report on the UI dashboard.

  - `result_counts: object { errored, failed, passed, total }`

    Counters summarizing the outcomes of the evaluation run.

    - `errored: number`

      Number of output items that resulted in an error.

    - `failed: number`

      Number of output items that failed to pass the evaluation.

    - `passed: number`

      Number of output items that passed the evaluation.

    - `total: number`

      Total number of executed output items.

  - `status: string`

    The status of the evaluation run.

### Run Retrieve Response

- `RunRetrieveResponse object { id, created_at, data_source, 11 more }`

  A schema representing an evaluation run.

  - `id: string`

    Unique identifier for the evaluation run.

  - `created_at: number`

    Unix timestamp (in seconds) when the evaluation run was created.

  - `data_source: CreateEvalJSONLRunDataSource or CreateEvalCompletionsRunDataSource or ResponsesRunDataSource { source, type, input_messages, 2 more }`

    Information about the run's data source.

    - `CreateEvalJSONLRunDataSource object { source, type }`

      A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }`

        Determines what populates the `item` namespace in the data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

      - `type: "jsonl"`

        The type of data source. Always `jsonl`.

        - `"jsonl"`

    - `CreateEvalCompletionsRunDataSource object { source, type, input_messages, 2 more }`

      A CompletionsRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or StoredCompletionsRunDataSource { type, created_after, created_before, 3 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `StoredCompletionsRunDataSource object { type, created_after, created_before, 3 more }`

          A StoredCompletionsRunDataSource configuration describing a set of filters

          - `type: "stored_completions"`

            The type of source. Always `stored_completions`.

            - `"stored_completions"`

          - `created_after: optional number or null`

            An optional Unix timestamp to filter items created after this time.

          - `created_before: optional number or null`

            An optional Unix timestamp to filter items created before this time.

          - `limit: optional number or null`

            An optional maximum number of items to return.

          - `metadata: optional Metadata or null`

            Set of 16 key-value pairs that can be attached to an object. This can be
            useful for storing additional information about the object in a structured
            format, and querying for objects via API or the dashboard.

            Keys are strings with a maximum length of 64 characters. Values are strings
            with a maximum length of 512 characters.

          - `model: optional string or null`

            An optional model to filter by (e.g., 'gpt-6-astra').

      - `type: "completions"`

        The type of run data source. Always `completions`.

        - `"completions"`

      - `input_messages: optional TemplateInputMessages { template, type }  or ItemReferenceInputMessages { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `TemplateInputMessages object { template, type }`

          - `template: array of EasyInputMessage or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `EasyInputMessage object { content, role, phase, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputMessageContentList`

                Text, image, or audio input to the model, used to generate a response.
                Can also contain previous assistant responses.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputMessageContentList = array of ResponseInputContent`

                  A list of one or many input items to the model, containing different content
                  types.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                    - `text: string`

                      The text input to the model.

                    - `type: "input_text"`

                      The type of the input item. Always `input_text`.

                      - `"input_text"`

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputImage object { detail, type, file_id, 2 more }`

                    An image input to the model. Learn about [image inputs](/api/docs/guides/images-vision).

                    - `detail: ImageDetail`

                      The detail level of the image to be sent to the model. One of `high`, `low`, `auto`, or `original`. Defaults to `auto`.

                      - `"low"`

                      - `"high"`

                      - `"auto"`

                      - `"original"`

                    - `type: "input_image"`

                      The type of the input item. Always `input_image`.

                      - `"input_image"`

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `image_url: optional string or null`

                      The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

                  - `ResponseInputFile object { type, detail, file_data, 4 more }`

                    A file input to the model.

                    - `type: "input_file"`

                      The type of the input item. Always `input_file`.

                      - `"input_file"`

                    - `detail: optional "auto" or "low" or "high"`

                      The detail level of the file to be sent to the model. Use `auto` to let the system select the detail level; for GPT-5.6 and later models, `auto` uses high-quality rendering, which may increase input token usage. Use `low` for lower-cost rendering, or `high` to render the file at higher quality. Defaults to `auto`.

                      - `"auto"`

                      - `"low"`

                      - `"high"`

                    - `file_data: optional string`

                      The content of the file to be sent to the model.

                    - `file_id: optional string or null`

                      The ID of the file to be sent to the model.

                    - `file_url: optional string`

                      The URL of the file to be sent to the model.

                    - `filename: optional string`

                      The name of the file to be sent to the model.

                    - `prompt_cache_breakpoint: optional object { mode }`

                      Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's `prompt_cache_options.ttl`; the boundary is not rounded to a token block.

                      - `mode: "explicit"`

                        The breakpoint mode. Always `explicit`.

                        - `"explicit"`

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `phase: optional "commentary" or "final_answer" or null`

                Labels an `assistant` message as intermediate commentary (`commentary`) or the final answer (`final_answer`).
                For models like `gpt-5.3-codex` and beyond, when sending follow-up requests, preserve and resend
                phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

                - `"commentary"`

                - `"final_answer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                  - `input_audio: object { data, format }`

                    - `data: string`

                      Base64-encoded audio data.

                    - `format: "mp3" or "wav"`

                      The format of the audio data. Currently supported formats are `mp3` and
                      `wav`.

                      - `"mp3"`

                      - `"wav"`

                  - `type: "input_audio"`

                    The type of the input item. Always `input_audio`.

                    - `"input_audio"`

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

                  - `TextInput = string`

                    A text input to the model.

                  - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                    A text input to the model.

                  - `OutputText object { text, type }`

                    A text output from the model.

                    - `text: string`

                      The text output from the model.

                    - `type: "output_text"`

                      The type of the output text. Always `output_text`.

                      - `"output_text"`

                  - `InputImage object { image_url, type, detail }`

                    An image input block used within EvalItem content arrays.

                    - `image_url: string`

                      The URL of the image input.

                    - `type: "input_image"`

                      The type of the image input. Always `input_image`.

                      - `"input_image"`

                    - `detail: optional string`

                      The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                  - `ResponseInputAudio object { input_audio, type }`

                    An audio input to the model.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `ItemReferenceInputMessages object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.input_trajectory"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, response_format, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

          - `"none"`

          - `"minimal"`

          - `"low"`

          - `"medium"`

          - `"high"`

          - `"xhigh"`

          - `"max"`

        - `response_format: optional ResponseFormatText or ResponseFormatJSONSchema or ResponseFormatJSONObject`

          An object specifying the format that the model must output.

          Setting to `{ "type": "json_schema", "json_schema": {...} }` enables
          Structured Outputs which ensures the model will match your supplied JSON
          schema. Learn more in the [Structured Outputs
          guide](/api/docs/guides/structured-outputs).

          Setting to `{ "type": "json_object" }` enables the older JSON mode, which
          ensures the message the model generates is valid JSON. Using `json_schema`
          is preferred for models that support it.

          - `ResponseFormatText object { type }`

            Default response format. Used to generate text responses.

            - `type: "text"`

              The type of response format being defined. Always `text`.

              - `"text"`

          - `ResponseFormatJSONSchema object { json_schema, type }`

            JSON Schema response format. Used to generate structured JSON responses.
            Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

            - `json_schema: object { name, description, schema, strict }`

              Structured Outputs configuration options, including a JSON Schema.

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `schema: optional map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `type: "json_schema"`

              The type of response format being defined. Always `json_schema`.

              - `"json_schema"`

          - `ResponseFormatJSONObject object { type }`

            JSON object response format. An older method of generating JSON responses.
            Using `json_schema` is recommended for models that support it. Note that the
            model will not generate JSON without a system or user message instructing it
            to do so.

            - `type: "json_object"`

              The type of response format being defined. Always `json_object`.

              - `"json_object"`

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `tools: optional array of ChatCompletionFunctionTool`

          A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

          - `function: FunctionDefinition`

            - `name: string`

              The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

            - `description: optional string`

              A description of what the function does, used by the model to choose when and how to call the function.

            - `parameters: optional FunctionParameters`

              The parameters the functions accepts, described as a JSON Schema object. See the [guide](/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format.

              Omitting `parameters` defines a function with an empty parameter list.

            - `strict: optional boolean or null`

              Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](/api/docs/guides/function-calling).

          - `type: "function"`

            The type of the tool. Currently, only `function` is supported.

            - `"function"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

    - `ResponsesRunDataSource object { source, type, input_messages, 2 more }`

      A ResponsesRunDataSource object describing a model sampling configuration.

      - `source: EvalJSONLFileContentSource { content, type }  or EvalJSONLFileIDSource { id, type }  or EvalResponsesSource { type, created_after, created_before, 8 more }`

        Determines what populates the `item` namespace in this run's data source.

        - `EvalJSONLFileContentSource object { content, type }`

          - `content: array of object { item, sample }`

            The content of the jsonl file.

            - `item: map[unknown]`

            - `sample: optional map[unknown]`

          - `type: "file_content"`

            The type of jsonl source. Always `file_content`.

            - `"file_content"`

        - `EvalJSONLFileIDSource object { id, type }`

          - `id: string`

            The identifier of the file.

          - `type: "file_id"`

            The type of jsonl source. Always `file_id`.

            - `"file_id"`

        - `EvalResponsesSource object { type, created_after, created_before, 8 more }`

          A EvalResponsesSource object describing a run data source configuration.

          - `type: "responses"`

            The type of run data source. Always `responses`.

            - `"responses"`

          - `created_after: optional number or null`

            Only include items created after this timestamp (inclusive). This is a query parameter used to select responses.

          - `created_before: optional number or null`

            Only include items created before this timestamp (inclusive). This is a query parameter used to select responses.

          - `instructions_search: optional string or null`

            Optional string to search the 'instructions' field. This is a query parameter used to select responses.

          - `metadata: optional unknown or null`

            Metadata filter for the responses. This is a query parameter used to select responses.

          - `model: optional string or null`

            The name of the model to find responses for. This is a query parameter used to select responses.

          - `reasoning_effort: optional ReasoningEffort or null`

            Constrains effort on reasoning for reasoning models. Currently supported
            values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
            Reducing reasoning effort can result in faster responses and fewer tokens
            used on reasoning in a response. Not all reasoning models support every
            value. See the
            [reasoning guide](/api/docs/guides/reasoning)
            for model-specific support.

          - `temperature: optional number or null`

            Sampling temperature. This is a query parameter used to select responses.

          - `tools: optional array of string or null`

            List of tool names. This is a query parameter used to select responses.

          - `top_p: optional number or null`

            Nucleus sampling parameter. This is a query parameter used to select responses.

          - `users: optional array of string or null`

            List of user identifiers. This is a query parameter used to select responses.

      - `type: "responses"`

        The type of run data source. Always `responses`.

        - `"responses"`

      - `input_messages: optional InputMessagesTemplate { template, type }  or InputMessagesItemReference { item_reference, type }`

        Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.

        - `InputMessagesTemplate object { template, type }`

          - `template: array of ChatMessage { content, role }  or EvalMessageObject { content, role, type }`

            A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.

            - `ChatMessage object { content, role }`

              - `content: string`

                The content of the message.

              - `role: string`

                The role of the message (e.g. "system", "assistant", "user").

            - `EvalMessageObject object { content, role, type }`

              A message input to the model with a role indicating instruction following
              hierarchy. Instructions given with the `developer` or `system` role take
              precedence over instructions given with the `user` role. Messages with the
              `assistant` role are presumed to have been generated by the model in previous
              interactions.

              - `content: string or ResponseInputText or OutputText { text, type }  or 3 more`

                Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

                - `TextInput = string`

                  A text input to the model.

                - `ResponseInputText object { text, type, prompt_cache_breakpoint }`

                  A text input to the model.

                - `OutputText object { text, type }`

                  A text output from the model.

                  - `text: string`

                    The text output from the model.

                  - `type: "output_text"`

                    The type of the output text. Always `output_text`.

                    - `"output_text"`

                - `InputImage object { image_url, type, detail }`

                  An image input block used within EvalItem content arrays.

                  - `image_url: string`

                    The URL of the image input.

                  - `type: "input_image"`

                    The type of the image input. Always `input_image`.

                    - `"input_image"`

                  - `detail: optional string`

                    The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.

                - `ResponseInputAudio object { input_audio, type }`

                  An audio input to the model.

                - `GraderInputs = array of string or ResponseInputText or OutputText { text, type }  or 2 more`

                  A list of inputs, each of which may be either an input text, output text, input
                  image, or input audio object.

              - `role: "user" or "assistant" or "system" or "developer"`

                The role of the message input. One of `user`, `assistant`, `system`, or
                `developer`.

                - `"user"`

                - `"assistant"`

                - `"system"`

                - `"developer"`

              - `type: optional "message"`

                The type of the message input. Always `message`.

                - `"message"`

          - `type: "template"`

            The type of input messages. Always `template`.

            - `"template"`

        - `InputMessagesItemReference object { item_reference, type }`

          - `item_reference: string`

            A reference to a variable in the `item` namespace. Ie, "item.name"

          - `type: "item_reference"`

            The type of input messages. Always `item_reference`.

            - `"item_reference"`

      - `model: optional string`

        The name of the model to use for generating completions (e.g. "o3-mini").

      - `sampling_params: optional object { max_completion_tokens, reasoning_effort, seed, 4 more }`

        - `max_completion_tokens: optional number`

          The maximum number of tokens in the generated output.

        - `reasoning_effort: optional ReasoningEffort or null`

          Constrains effort on reasoning for reasoning models. Currently supported
          values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`.
          Reducing reasoning effort can result in faster responses and fewer tokens
          used on reasoning in a response. Not all reasoning models support every
          value. See the
          [reasoning guide](/api/docs/guides/reasoning)
          for model-specific support.

        - `seed: optional number`

          A seed value to initialize the randomness, during sampling.

        - `temperature: optional number`

          A higher temperature increases randomness in the outputs.

        - `text: optional object { format }`

          Configuration options for a text response from the model. Can be plain
          text or structured JSON data. Learn more:

          - [Text inputs and outputs](/api/docs/guides/text)
          - [Structured Outputs](/api/docs/guides/structured-outputs)

          - `format: optional ResponseFormatTextConfig`

            An object specifying the format that the model must output.

            Configuring `{ "type": "json_schema" }` enables Structured Outputs,
            which ensures the model will match your supplied JSON schema. Learn more in the
            [Structured Outputs guide](/api/docs/guides/structured-outputs).

            The default format is `{ "type": "text" }` with no additional options.

            **Not recommended for gpt-4o and newer models:**

            Setting to `{ "type": "json_object" }` enables the older JSON mode, which
            ensures the message the model generates is valid JSON. Using `json_schema`
            is preferred for models that support it.

            - `ResponseFormatText object { type }`

              Default response format. Used to generate text responses.

            - `ResponseFormatTextJSONSchemaConfig object { name, schema, type, 2 more }`

              JSON Schema response format. Used to generate structured JSON responses.
              Learn more about [Structured Outputs](/api/docs/guides/structured-outputs).

              - `name: string`

                The name of the response format. Must be a-z, A-Z, 0-9, or contain
                underscores and dashes, with a maximum length of 64.

              - `schema: map[unknown]`

                The schema for the response format, described as a JSON Schema object.
                Learn how to build JSON schemas [here](https://json-schema.org/).

              - `type: "json_schema"`

                The type of response format being defined. Always `json_schema`.

                - `"json_schema"`

              - `description: optional string`

                A description of what the response format is for, used by the model to
                determine how to respond in the format.

              - `strict: optional boolean or null`

                Whether to enable strict schema adherence when generating the output.
                If set to true, the model will always follow the exact schema defined
                in the `schema` field. Only a subset of JSON Schema is supported when
                `strict` is `true`. To learn more, read the [Structured Outputs
                guide](/api/docs/guides/structured-outputs).

            - `ResponseFormatJSONObject object { type }`

              JSON object response format. An older method of generating JSON responses.
              Using `json_schema` is recommended for models that support it. Note that the
              model will not generate JSON without a system or user message instructing it
              to do so.

        - `tools: optional array of Function { name, parameters, strict, 6 more }  or FileSearch { type, vector_store_ids, filters, 2 more }  or Computer { type }  or 13 more`

          An array of tools the model may call while generating a response. You
          can specify which tool to use by setting the `tool_choice` parameter.

          The two categories of tools you can provide the model are:

          - **Built-in tools**: Tools that are provided by OpenAI that extend the
            model's capabilities, like [web search](/api/docs/guides/tools-web-search)
            or [file search](/api/docs/guides/tools-file-search). Learn more about
            [built-in tools](/api/docs/guides/tools).
          - **Function calls (custom tools)**: Functions that are defined by you,
            enabling the model to call your own code. Learn more about
            [function calling](/api/docs/guides/function-calling).

          - `Function object { name, parameters, strict, 6 more }`

            Defines a function in your own code the model can choose to call. Learn more about [function calling](/api/docs/guides/function-calling).

            - `name: string`

              The name of the function to call.

            - `parameters: map[unknown] or null`

              A JSON schema object describing the parameters of the function.

            - `strict: boolean or null`

              Whether strict parameter validation is enforced for this function tool.

            - `type: "function"`

              The type of the function tool. Always `function`.

              - `"function"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

            - `defer_loading: optional boolean`

              Whether this function is deferred and loaded via tool search.

            - `description: optional string or null`

              A description of the function. Used by the model to determine whether or not to call the function.

            - `output_schema: optional map[unknown] or null`

              A JSON schema object describing the JSON value encoded in string outputs for this function.

          - `FileSearch object { type, vector_store_ids, filters, 2 more }`

            A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](/api/docs/guides/tools-file-search).

            - `type: "file_search"`

              The type of the file search tool. Always `file_search`.

              - `"file_search"`

            - `vector_store_ids: array of string`

              The IDs of the vector stores to search.

            - `filters: optional ComparisonFilter or CompoundFilter or null`

              A filter to apply.

              - `ComparisonFilter object { key, type, value }`

                A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                - `key: string`

                  The key to compare against the value.

                - `type: "eq" or "ne" or "gt" or 5 more`

                  Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`.

                  - `eq`: equals
                  - `ne`: not equal
                  - `gt`: greater than
                  - `gte`: greater than or equal
                  - `lt`: less than
                  - `lte`: less than or equal
                  - `in`: in
                  - `nin`: not in

                  - `"eq"`

                  - `"ne"`

                  - `"gt"`

                  - `"gte"`

                  - `"lt"`

                  - `"lte"`

                  - `"in"`

                  - `"nin"`

                - `value: string or number or boolean or array of string or number`

                  The value to compare against the attribute key; supports string, number, or boolean types.

                  - `string`

                  - `number`

                  - `boolean`

                  - `array of string or number`

                    - `string`

                    - `number`

              - `CompoundFilter object { filters, type }`

                Combine multiple filters using `and` or `or`.

                - `filters: array of ComparisonFilter or CompoundFilter`

                  Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.

                  - `ComparisonFilter object { key, type, value }`

                    A filter used to compare a specified attribute key to a given value using a defined comparison operation.

                  - `CompoundFilter object { filters, type }`

                    Combine multiple filters using `and` or `or`.

                - `type: "and" or "or"`

                  Type of operation: `and` or `or`.

                  - `"and"`

                  - `"or"`

            - `max_num_results: optional number`

              The maximum number of results to return. This number should be between 1 and 50 inclusive.

            - `ranking_options: optional object { hybrid_search, ranker, score_threshold }`

              Ranking options for search.

              - `hybrid_search: optional object { embedding_weight, text_weight }`

                Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

                - `embedding_weight: number`

                  The weight of the embedding in the reciprocal ranking fusion.

                - `text_weight: number`

                  The weight of the text in the reciprocal ranking fusion.

              - `ranker: optional "auto" or "default-2024-11-15"`

                The ranker to use for the file search.

                - `"auto"`

                - `"default-2024-11-15"`

              - `score_threshold: optional number`

                The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

          - `Computer object { type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `type: "computer"`

              The type of the computer tool. Always `computer`.

              - `"computer"`

          - `ComputerUsePreview object { display_height, display_width, environment, type }`

            A tool that controls a virtual computer. Learn more about the [computer tool](/api/docs/guides/tools-computer-use).

            - `display_height: number`

              The height of the computer display.

            - `display_width: number`

              The width of the computer display.

            - `environment: "windows" or "mac" or "linux" or 2 more`

              The type of computer environment to control.

              - `"windows"`

              - `"mac"`

              - `"linux"`

              - `"ubuntu"`

              - `"browser"`

            - `type: "computer_use_preview"`

              The type of the computer use tool. Always `computer_use_preview`.

              - `"computer_use_preview"`

          - `WebSearch object { type, external_web_access, filters, 2 more }`

            Search the Internet for sources related to the prompt. Learn more about the
            [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search" or "web_search_2025_08_26"`

              The type of the web search tool. One of `web_search` or `web_search_2025_08_26`.

              - `"web_search"`

              - `"web_search_2025_08_26"`

            - `external_web_access: optional boolean`

              Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

            - `filters: optional object { allowed_domains }  or null`

              Filters for the search.

              - `allowed_domains: optional array of string or null`

                Allowed domains for the search. If not provided, all domains are allowed.
                Subdomains of the provided domains are allowed as well.

                Example: `["pubmed.ncbi.nlm.nih.gov"]`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { city, country, region, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the
              United States. To avoid this fallback, pass `{"type": "approximate"}` without
              location fields. To localize results, provide the relevant location fields.

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

              - `type: optional "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

          - `Mcp object { server_label, type, allowed_callers, 9 more }`

            Give the model access to additional tools via remote Model Context Protocol
            (MCP) servers. [Learn more about MCP](/api/docs/guides/tools-connectors-mcp).

            - `server_label: string`

              A label for this MCP server, used to identify it in tool calls.

            - `type: "mcp"`

              The type of the MCP tool. Always `mcp`.

              - `"mcp"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `allowed_tools: optional array of string or McpToolFilter { read_only, tool_names }  or null`

              List of allowed tool names or a filter object.

              - `McpAllowedTools = array of string`

                A string array of allowed tool names

              - `McpToolFilter object { read_only, tool_names }`

                A filter object to specify which tools are allowed.

                - `read_only: optional boolean`

                  Indicates whether or not a tool modifies data or is read-only. If an
                  MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                  it will match this filter.

                - `tool_names: optional array of string`

                  List of allowed tool names.

            - `authorization: optional string`

              An OAuth access token that can be used with a remote MCP server, either
              with a custom MCP server URL or a service connector. Your application
              must handle the OAuth authorization flow and provide the token here.

            - `connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more`

              Identifier for service connectors, like those available in ChatGPT. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided. Learn more
              about service connectors [here](/api/docs/guides/tools-connectors-mcp#connectors).

              This field is deprecated for models released after September 1, 2026.
              Use `server_url` to connect to a remote MCP server, or `tunnel_id` to
              connect through a Secure MCP Tunnel.

              Currently supported `connector_id` values are:

              - Dropbox: `connector_dropbox`
              - Gmail: `connector_gmail`
              - Google Calendar: `connector_googlecalendar`
              - Google Drive: `connector_googledrive`
              - Microsoft Teams: `connector_microsoftteams`
              - Outlook Calendar: `connector_outlookcalendar`
              - Outlook Email: `connector_outlookemail`
              - SharePoint: `connector_sharepoint`

              - `"connector_dropbox"`

              - `"connector_gmail"`

              - `"connector_googlecalendar"`

              - `"connector_googledrive"`

              - `"connector_microsoftteams"`

              - `"connector_outlookcalendar"`

              - `"connector_outlookemail"`

              - `"connector_sharepoint"`

            - `defer_loading: optional boolean`

              Whether this MCP tool is deferred and discovered via tool search.

            - `headers: optional map[string] or null`

              Optional HTTP headers to send to the MCP server. Use for authentication
              or other purposes.

            - `require_approval: optional McpToolApprovalFilter { always, never }  or "always" or "never" or null`

              Specify which of the MCP server's tools require approval.

              - `McpToolApprovalFilter object { always, never }`

                Specify which of the MCP server's tools require approval. Can be
                `always`, `never`, or a filter object associated with tools
                that require approval.

                - `always: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

                - `never: optional object { read_only, tool_names }`

                  A filter object to specify which tools are allowed.

                  - `read_only: optional boolean`

                    Indicates whether or not a tool modifies data or is read-only. If an
                    MCP server is [annotated with `readOnlyHint`](https://modelcontextprotocol.io/specification/2025-06-18/schema#toolannotations-readonlyhint),
                    it will match this filter.

                  - `tool_names: optional array of string`

                    List of allowed tool names.

              - `McpToolApprovalSetting = "always" or "never"`

                Specify a single approval policy for all tools. One of `always` or
                `never`. When set to `always`, all tools will require approval. When
                set to `never`, all tools will not require approval.

                - `"always"`

                - `"never"`

            - `server_description: optional string`

              Optional description of the MCP server, used to provide more context.

            - `server_url: optional string`

              The URL for the MCP server. One of `server_url`, `connector_id`, or
              `tunnel_id` must be provided.

            - `tunnel_id: optional string`

              The Secure MCP Tunnel ID to use instead of a direct server URL. One of
              `server_url`, `connector_id`, or `tunnel_id` must be provided.

          - `CodeInterpreter object { container, type, allowed_callers }`

            A tool that runs Python code to help generate a response to a prompt.

            - `container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }`

              The code interpreter container. Can be a container ID or an object that
              specifies uploaded file IDs to make available to your code, along with an
              optional `memory_limit` setting.

              - `string`

                The container ID.

              - `CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }`

                Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

                - `type: "auto"`

                  Always `auto`.

                  - `"auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the code interpreter container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                    - `type: "disabled"`

                      Disable outbound network access. Always `disabled`.

                      - `"disabled"`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                    - `allowed_domains: array of string`

                      A list of allowed domains when type is `allowlist`.

                    - `type: "allowlist"`

                      Allow outbound network access only to specified domains. Always `allowlist`.

                      - `"allowlist"`

                    - `domain_secrets: optional array of ContainerNetworkPolicyDomainSecret`

                      Optional domain-scoped secrets for allowlisted domains.

                      - `domain: string`

                        The domain associated with the secret.

                      - `name: string`

                        The name of the secret to inject for the domain.

                      - `value: string`

                        The secret value to inject for the domain.

            - `type: "code_interpreter"`

              The type of the code interpreter tool. Always `code_interpreter`.

              - `"code_interpreter"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

          - `ProgrammaticToolCalling object { type }`

            - `type: "programmatic_tool_calling"`

              The type of the tool. Always `programmatic_tool_calling`.

              - `"programmatic_tool_calling"`

          - `ImageGeneration object { type, action, background, 9 more }`

            A tool that generates images using the GPT image models.

            - `type: "image_generation"`

              The type of the image generation tool. Always `image_generation`.

              - `"image_generation"`

            - `action: optional "generate" or "edit" or "auto"`

              Whether to generate a new image or edit an existing image. Default: `auto`.

              - `"generate"`

              - `"edit"`

              - `"auto"`

            - `background: optional "transparent" or "opaque" or "auto"`

              Set the background of the generated image. One of `transparent`, `opaque`,
              or `auto`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`, including
              their `2026-09-08` snapshots, support `opaque` and `transparent`
              backgrounds. Transparent backgrounds are available for supported GPT Image
              models. For `gpt-image-2` and `gpt-image-2-2026-04-21`, this support is in
              preview. When using `transparent`, set the output format to `png` or `webp`.
              Default: `auto`.

              - `"transparent"`

              - `"opaque"`

              - `"auto"`

            - `input_fidelity: optional "high" or "low" or null`

              Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports `high` and `low` on `gpt-image-1` and `gpt-image-1.5`; `gpt-image-1-mini` supports only `low`. For `gpt-image-2`, omit this parameter. Defaults to `low` on supported models.

              - `"high"`

              - `"low"`

            - `input_image_mask: optional object { file_id, image_url }`

              Optional mask for inpainting. Contains `image_url`
              (string, optional) and `file_id` (string, optional).

              - `file_id: optional string`

                File ID for the mask image.

              - `image_url: optional string`

                Base64-encoded mask image.

            - `model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

              The image generation model to use. One of `gpt-image-1`,
              `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
              `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
              `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
              `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
              `gpt-image-1`.

              - `string`

              - `"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more`

                The image generation model to use. One of `gpt-image-1`,
                `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2`,
                `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`,
                `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`,
                `gpt-image-2.5-flare-2026-09-08`, or `chatgpt-image-latest`. Default:
                `gpt-image-1`.

                - `"gpt-image-1"`

                - `"gpt-image-1-mini"`

                - `"gpt-image-1.5"`

                - `"gpt-image-2"`

                - `"gpt-image-2-2026-04-21"`

                - `"gpt-image-2.5-sunburst"`

                - `"gpt-image-2.5-sunburst-2026-09-08"`

                - `"gpt-image-2.5-flare"`

                - `"gpt-image-2.5-flare-2026-09-08"`

            - `moderation: optional "auto" or "low"`

              Moderation level for the generated image. Default: `auto`.

              - `"auto"`

              - `"low"`

            - `output_compression: optional number`

              Compression level for the output image. Default: 100.

            - `output_format: optional "png" or "webp" or "jpeg"`

              The output format of the generated image. One of `png`, `webp`, or
              `jpeg`. Default: `png`.

              - `"png"`

              - `"webp"`

              - `"jpeg"`

            - `partial_images: optional number`

              Number of partial images to generate in streaming mode, from 0 (default value) to 3.

            - `quality: optional "low" or "medium" or "high" or 3 more`

              The quality of the generated image. The GPT image models support `low`,
              `medium`, and `high`. `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare`,
              including their `2026-09-08` snapshots, also support `xhigh` and `max`.
              Default: `auto`.

              - `"low"`

              - `"medium"`

              - `"high"`

              - `"xhigh"`

              - `"max"`

              - `"auto"`

            - `size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"`

              The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

              - `string`

              - `"1024x1024" or "1024x1536" or "1536x1024" or "auto"`

                The size of the generated images. For `gpt-image-2`, `gpt-image-2-2026-04-21`, `gpt-image-2.5-sunburst`, `gpt-image-2.5-sunburst-2026-09-08`, `gpt-image-2.5-flare`, and `gpt-image-2.5-flare-2026-09-08`, arbitrary resolutions are supported as `WIDTHxHEIGHT` strings, for example `1536x864`. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above `2560x1440` are experimental, and the maximum supported resolution is `3840x2160`. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes `1024x1024`, `1536x1024`, and `1024x1536` are supported by the GPT image models; `auto` is supported for models that allow automatic sizing.

                - `"1024x1024"`

                - `"1024x1536"`

                - `"1536x1024"`

                - `"auto"`

          - `LocalShell object { type }`

            A tool that allows the model to execute shell commands in a local environment.

            - `type: "local_shell"`

              The type of the local shell tool. Always `local_shell`.

              - `"local_shell"`

          - `Shell object { type, allowed_callers, environment }`

            A tool that allows the model to execute shell commands.

            - `type: "shell"`

              The type of the shell tool. Always `shell`.

              - `"shell"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `environment: optional ContainerAuto or LocalEnvironment or ContainerReference or null`

              - `ContainerAuto object { type, file_ids, memory_limit, 2 more }`

                - `type: "container_auto"`

                  Automatically creates a container for this request

                  - `"container_auto"`

                - `file_ids: optional array of string`

                  An optional list of uploaded files to make available to your code.

                - `memory_limit: optional "1g" or "4g" or "16g" or "64g" or null`

                  The memory limit for the container.

                  - `"1g"`

                  - `"4g"`

                  - `"16g"`

                  - `"64g"`

                - `network_policy: optional ContainerNetworkPolicyDisabled or ContainerNetworkPolicyAllowlist`

                  Network access policy for the container.

                  - `ContainerNetworkPolicyDisabled object { type }`

                  - `ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }`

                - `skills: optional array of SkillReference or InlineSkill`

                  An optional list of skills referenced by id or inline data.

                  - `SkillReference object { skill_id, type, version }`

                    - `skill_id: string`

                      The ID of the referenced skill.

                    - `type: "skill_reference"`

                      References a skill created with the /v1/skills endpoint.

                      - `"skill_reference"`

                    - `version: optional string`

                      Optional skill version. Use a positive integer or 'latest'. Omit for default.

                  - `InlineSkill object { description, name, source, type }`

                    - `description: string`

                      The description of the skill.

                    - `name: string`

                      The name of the skill.

                    - `source: InlineSkillSource`

                      Inline skill payload

                      - `data: string`

                        Base64-encoded skill zip bundle.

                      - `media_type: "application/zip"`

                        The media type of the inline skill payload. Must be `application/zip`.

                        - `"application/zip"`

                      - `type: "base64"`

                        The type of the inline skill source. Must be `base64`.

                        - `"base64"`

                    - `type: "inline"`

                      Defines an inline skill for this request.

                      - `"inline"`

              - `LocalEnvironment object { type, skills }`

                - `type: "local"`

                  Use a local computer environment.

                  - `"local"`

                - `skills: optional array of LocalSkill`

                  An optional list of skills.

                  - `description: string`

                    The description of the skill.

                  - `name: string`

                    The name of the skill.

                  - `path: string`

                    The path to the directory containing the skill.

              - `ContainerReference object { container_id, type }`

                - `container_id: string`

                  The ID of the referenced container.

                - `type: "container_reference"`

                  References a container created with the /v1/containers endpoint

                  - `"container_reference"`

          - `Custom object { name, type, allowed_callers, 4 more }`

            A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

            - `name: string`

              The name of the custom tool, used to identify it in tool calls.

            - `type: "custom"`

              The type of the custom tool. Always `custom`.

              - `"custom"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

            - `async: optional boolean`

              Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

            - `defer_loading: optional boolean`

              Whether this tool should be deferred and discovered via tool search.

            - `description: optional string`

              Optional description of the custom tool, used to provide more context.

            - `format: optional CustomToolInputFormat`

              The input format for the custom tool. Default is unconstrained text.

              - `Text object { type }`

                Unconstrained free-form text.

                - `type: "text"`

                  Unconstrained text format. Always `text`.

                  - `"text"`

              - `Grammar object { definition, syntax, type }`

                A grammar defined by the user.

                - `definition: string`

                  The grammar definition.

                - `syntax: "lark" or "regex"`

                  The syntax of the grammar definition. One of `lark` or `regex`.

                  - `"lark"`

                  - `"regex"`

                - `type: "grammar"`

                  Grammar format. Always `grammar`.

                  - `"grammar"`

          - `Namespace object { description, name, tools, type }`

            Groups function/custom tools under a shared namespace.

            - `description: string`

              A description of the namespace shown to the model.

            - `name: string`

              The namespace name used in tool calls (for example, `crm`).

            - `tools: array of Function { name, type, allowed_callers, 6 more }  or Custom { name, type, allowed_callers, 4 more }`

              The function/custom tools available inside this namespace.

              - `Function object { name, type, allowed_callers, 6 more }`

                - `name: string`

                - `type: "function"`

                  - `"function"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this function should be deferred and discovered via tool search.

                - `description: optional string or null`

                - `output_schema: optional map[unknown] or null`

                  A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

                - `parameters: optional unknown or null`

                - `strict: optional boolean or null`

                  Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

              - `Custom object { name, type, allowed_callers, 4 more }`

                A custom tool that processes input using a specified format. Learn more about   [custom tools](/api/docs/guides/function-calling#custom-tools)

                - `name: string`

                  The name of the custom tool, used to identify it in tool calls.

                - `type: "custom"`

                  The type of the custom tool. Always `custom`.

                  - `"custom"`

                - `allowed_callers: optional array of "direct" or "programmatic" or null`

                  The tool invocation context(s).

                  - `"direct"`

                  - `"programmatic"`

                - `async: optional boolean`

                  Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

                - `defer_loading: optional boolean`

                  Whether this tool should be deferred and discovered via tool search.

                - `description: optional string`

                  Optional description of the custom tool, used to provide more context.

                - `format: optional CustomToolInputFormat`

                  The input format for the custom tool. Default is unconstrained text.

            - `type: "namespace"`

              The type of the tool. Always `namespace`.

              - `"namespace"`

          - `ToolSearch object { type, description, execution, parameters }`

            Hosted or BYOT tool search configuration for deferred tools.

            - `type: "tool_search"`

              The type of the tool. Always `tool_search`.

              - `"tool_search"`

            - `description: optional string or null`

              Description shown to the model for a client-executed tool search tool.

            - `execution: optional "server" or "client"`

              Whether tool search is executed by the server or by the client.

              - `"server"`

              - `"client"`

            - `parameters: optional unknown or null`

              Parameter schema for a client-executed tool search tool.

          - `WebSearchPreview object { type, search_content_types, search_context_size, user_location }`

            This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](/api/docs/guides/tools-web-search).

            - `type: "web_search_preview" or "web_search_preview_2025_03_11"`

              The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`.

              - `"web_search_preview"`

              - `"web_search_preview_2025_03_11"`

            - `search_content_types: optional array of "text" or "image"`

              - `"text"`

              - `"image"`

            - `search_context_size: optional "low" or "medium" or "high"`

              High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default.

              - `"low"`

              - `"medium"`

              - `"high"`

            - `user_location: optional object { type, city, country, 2 more }  or null`

              The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass `{"type": "approximate"}` without location fields. To localize results, provide the relevant location fields.

              - `type: "approximate"`

                The type of location approximation. Always `approximate`.

                - `"approximate"`

              - `city: optional string or null`

                Free text input for the city of the user, e.g. `San Francisco`.

              - `country: optional string or null`

                The two-letter [ISO country code](https://en.wikipedia.org/wiki/ISO_3166-1) of the user, e.g. `US`.

              - `region: optional string or null`

                Free text input for the region of the user, e.g. `California`.

              - `timezone: optional string or null`

                The [IANA timezone](https://timeapi.io/documentation/iana-timezones) of the user, e.g. `America/Los_Angeles`.

          - `ApplyPatch object { type, allowed_callers }`

            Allows the assistant to create, delete, or update files using unified diffs.

            - `type: "apply_patch"`

              The type of the tool. Always `apply_patch`.

              - `"apply_patch"`

            - `allowed_callers: optional array of "direct" or "programmatic" or null`

              The tool invocation context(s).

              - `"direct"`

              - `"programmatic"`

        - `top_p: optional number`

          An alternative to temperature for nucleus sampling; 1.0 includes all tokens.

  - `error: EvalAPIError or null`

    An object representing an error response from the Eval API.

    - `code: string`

      The error code.

    - `message: string`

      The error message.

  - `eval_id: string`

    The identifier of the associated evaluation.

  - `metadata: Metadata or null`

    Set of 16 key-value pairs that can be attached to an object. This can be
    useful for storing additional information about the object in a structured
    format, and querying for objects via API or the dashboard.

    Keys are strings with a maximum length of 64 characters. Values are strings
    with a maximum length of 512 characters.

  - `model: string or null`

    The model that is evaluated, if applicable.

  - `name: string or null`

    The name of the evaluation run.

  - `object: "eval.run"`

    The type of the object. Always "eval.run".

    - `"eval.run"`

  - `per_model_usage: array of object { cached_tokens, completion_tokens, invocation_count, 3 more }  or null`

    Usage statistics for each model during the evaluation run.

    - `cached_tokens: number`

      The number of tokens retrieved from cache.

    - `completion_tokens: number`

      The number of completion tokens generated.

    - `invocation_count: number`

      The number of invocations.

    - `model_name: string`

      The name of the model.

    - `prompt_tokens: number`

      The number of prompt tokens used.

    - `total_tokens: number`

      The total number of tokens used.

  - `per_testing_criteria_results: array of object { failed, passed, testing_criteria }  or null`

    Results per testing criteria applied during the evaluation run.

    - `failed: number`

      Number of tests failed for this criteria.

    - `passed: number`

      Number of tests passed for this criteria.

    - `testing_criteria: string`

      A description of the testing criteria.

  - `report_url: string`

    The URL to the rendered evaluation run report on the UI dashboard.

  - `result_counts: object { errored, failed, passed, total }`

    Counters summarizing the outcomes of the evaluation run.

    - `errored: number`

      Number of output items that resulted in an error.

    - `failed: number`

      Number of output items that failed to pass the evaluation.

    - `passed: number`

      Number of output items that passed the evaluation.

    - `total: number`

      Total number of executed output items.

  - `status: string`

    The status of the evaluation run.

# Output Items

## Get eval run output items

**get** `/evals/{eval_id}/runs/{run_id}/output_items`

Get a list of output items for an evaluation run.

### Path Parameters

- `eval_id: string`

- `run_id: string`

### Query Parameters

- `after: optional string`

  Identifier for the last output item from the previous pagination request.

- `limit: optional number`

  Number of output items to retrieve.

- `order: optional "asc" or "desc"`

  Sort order for output items by timestamp. Use `asc` for ascending order or `desc` for descending order. Defaults to `asc`.

  - `"asc"`

  - `"desc"`

- `status: optional "fail" or "pass"`

  Filter output items by status. Use `failed` to filter by failed output
  items or `pass` to filter by passed output items.

  - `"fail"`

  - `"pass"`

### Returns

- `data: array of object { id, created_at, datasource_item, 7 more }`

  An array of eval run output item objects.

  - `id: string`

    Unique identifier for the evaluation run output item.

  - `created_at: number`

    Unix timestamp (in seconds) when the evaluation run was created.

  - `datasource_item: map[unknown]`

    Details of the input data source item.

  - `datasource_item_id: number`

    The identifier for the data source item.

  - `eval_id: string`

    The identifier of the evaluation group.

  - `object: "eval.run.output_item"`

    The type of the object. Always "eval.run.output_item".

    - `"eval.run.output_item"`

  - `results: array of object { name, passed, score, 2 more }`

    A list of grader results for this output item.

    - `name: string`

      The name of the grader.

    - `passed: boolean`

      Whether the grader considered the output a pass.

    - `score: number`

      The numeric score produced by the grader.

    - `sample: optional map[unknown] or null`

      Optional sample or intermediate data produced by the grader.

    - `type: optional string`

      The grader type (for example, "string-check-grader").

  - `run_id: string`

    The identifier of the evaluation run associated with this output item.

  - `sample: object { error, finish_reason, input, 7 more }`

    A sample containing the input and output of the evaluation run.

    - `error: EvalAPIError or null`

      An object representing an error response from the Eval API.

      - `code: string`

        The error code.

      - `message: string`

        The error message.

    - `finish_reason: string`

      The reason why the sample generation was finished.

    - `input: array of object { content, role }`

      An array of input messages.

      - `content: string`

        The content of the message.

      - `role: string`

        The role of the message sender (e.g., system, user, developer).

    - `max_completion_tokens: number`

      The maximum number of tokens allowed for completion.

    - `model: string`

      The model used for generating the sample.

    - `output: array of object { content, role }`

      An array of output messages.

      - `content: optional string`

        The content of the message.

      - `role: optional string`

        The role of the message (e.g. "system", "assistant", "user").

    - `seed: number`

      The seed used for generating the sample.

    - `temperature: number`

      The sampling temperature used.

    - `top_p: number`

      The top_p value used for sampling.

    - `usage: object { cached_tokens, completion_tokens, prompt_tokens, total_tokens }`

      Token usage details for the sample.

      - `cached_tokens: number`

        The number of tokens retrieved from cache.

      - `completion_tokens: number`

        The number of completion tokens generated.

      - `prompt_tokens: number`

        The number of prompt tokens used.

      - `total_tokens: number`

        The total number of tokens used.

  - `status: string`

    The status of the evaluation run.

- `first_id: string`

  The identifier of the first eval run output item in the data array.

- `has_more: boolean`

  Indicates whether there are more eval run output items available.

- `last_id: string`

  The identifier of the last eval run output item in the data array.

- `object: "list"`

  The type of this object. It is always set to "list".

  - `"list"`

### Example

```http
curl https://api.openai.com/v1/evals/$EVAL_ID/runs/$RUN_ID/output_items \
    -H "Authorization: Bearer $OPENAI_API_KEY"
```

#### Response

```json
{
  "data": [
    {
      "id": "id",
      "created_at": 0,
      "datasource_item": {
        "foo": "bar"
      },
      "datasource_item_id": 0,
      "eval_id": "eval_id",
      "object": "eval.run.output_item",
      "results": [
        {
          "name": "name",
          "passed": true,
          "score": 0,
          "sample": {
            "foo": "bar"
          },
          "type": "type"
        }
      ],
      "run_id": "run_id",
      "sample": {
        "error": {
          "code": "code",
          "message": "message"
        },
        "finish_reason": "finish_reason",
        "input": [
          {
            "content": "content",
            "role": "role"
          }
        ],
        "max_completion_tokens": 0,
        "model": "model",
        "output": [
          {
            "content": "content",
            "role": "role"
          }
        ],
        "seed": 0,
        "temperature": 0,
        "top_p": 0,
        "usage": {
          "cached_tokens": 0,
          "completion_tokens": 0,
          "prompt_tokens": 0,
          "total_tokens": 0
        }
      },
      "status": "status"
    }
  ],
  "first_id": "first_id",
  "has_more": true,
  "last_id": "last_id",
  "object": "list"
}
```

### Example

```http
curl https://api.openai.com/v1/evals/egroup_67abd54d9b0081909a86353f6fb9317a/runs/erun_67abd54d60ec8190832b46859da808f7/output_items \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json"
```

#### Response

```json
{
  "object": "list",
  "data": [
    {
      "object": "eval.run.output_item",
      "id": "outputitem_67e5796c28e081909917bf79f6e6214d",
      "created_at": 1743092076,
      "run_id": "evalrun_67abd54d60ec8190832b46859da808f7",
      "eval_id": "eval_67abd54d9b0081909a86353f6fb9317a",
      "status": "pass",
      "datasource_item_id": 5,
      "datasource_item": {
        "input": "Stock Markets Rally After Positive Economic Data Released",
        "ground_truth": "Markets"
      },
      "results": [
        {
          "name": "String check-a2486074-d803-4445-b431-ad2262e85d47",
          "sample": null,
          "passed": true,
          "score": 1.0
        }
      ],
      "sample": {
        "input": [
          {
            "role": "developer",
            "content": "Categorize a given news headline into one of the following topics: Technology, Markets, World, Business, or Sports.\n\n# Steps\n\n1. Analyze the content of the news headline to understand its primary focus.\n2. Extract the subject matter, identifying any key indicators or keywords.\n3. Use the identified indicators to determine the most suitable category out of the five options: Technology, Markets, World, Business, or Sports.\n4. Ensure only one category is selected per headline.\n\n# Output Format\n\nRespond with the chosen category as a single word. For instance: \"Technology\", \"Markets\", \"World\", \"Business\", or \"Sports\".\n\n# Examples\n\n**Input**: \"Apple Unveils New iPhone Model, Featuring Advanced AI Features\"  \n**Output**: \"Technology\"\n\n**Input**: \"Global Stocks Mixed as Investors Await Central Bank Decisions\"  \n**Output**: \"Markets\"\n\n**Input**: \"War in Ukraine: Latest Updates on Negotiation Status\"  \n**Output**: \"World\"\n\n**Input**: \"Microsoft in Talks to Acquire Gaming Company for $2 Billion\"  \n**Output**: \"Business\"\n\n**Input**: \"Manchester United Secures Win in Premier League Football Match\"  \n**Output**: \"Sports\" \n\n# Notes\n\n- If the headline appears to fit into more than one category, choose the most dominant theme.\n- Keywords or phrases such as \"stocks\", \"company acquisition\", \"match\", or technological brands can be good indicators for classification.\n",
            "tool_call_id": null,
            "tool_calls": null,
            "function_call": null
          },
          {
            "role": "user",
            "content": "Stock Markets Rally After Positive Economic Data Released",
            "tool_call_id": null,
            "tool_calls": null,
            "function_call": null
          }
        ],
        "output": [
          {
            "role": "assistant",
            "content": "Markets",
            "tool_call_id": null,
            "tool_calls": null,
            "function_call": null
          }
        ],
        "finish_reason": "stop",
        "model": "gpt-6-astra",
        "usage": {
          "total_tokens": 325,
          "completion_tokens": 2,
          "prompt_tokens": 323,
          "cached_tokens": 0
        },
        "error": null,
        "temperature": 1.0,
        "max_completion_tokens": 2048,
        "top_p": 1.0,
        "seed": 42
      }
    }
  ],
  "first_id": "outputitem_67e5796c28e081909917bf79f6e6214d",
  "last_id": "outputitem_67e5796c28e081909917bf79f6e6214d",
  "has_more": true
}
```

## Get an output item of an eval run

**get** `/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}`

Get an evaluation run output item by ID.

### Path Parameters

- `eval_id: string`

- `run_id: string`

- `output_item_id: string`

### Returns

- `id: string`

  Unique identifier for the evaluation run output item.

- `created_at: number`

  Unix timestamp (in seconds) when the evaluation run was created.

- `datasource_item: map[unknown]`

  Details of the input data source item.

- `datasource_item_id: number`

  The identifier for the data source item.

- `eval_id: string`

  The identifier of the evaluation group.

- `object: "eval.run.output_item"`

  The type of the object. Always "eval.run.output_item".

  - `"eval.run.output_item"`

- `results: array of object { name, passed, score, 2 more }`

  A list of grader results for this output item.

  - `name: string`

    The name of the grader.

  - `passed: boolean`

    Whether the grader considered the output a pass.

  - `score: number`

    The numeric score produced by the grader.

  - `sample: optional map[unknown] or null`

    Optional sample or intermediate data produced by the grader.

  - `type: optional string`

    The grader type (for example, "string-check-grader").

- `run_id: string`

  The identifier of the evaluation run associated with this output item.

- `sample: object { error, finish_reason, input, 7 more }`

  A sample containing the input and output of the evaluation run.

  - `error: EvalAPIError or null`

    An object representing an error response from the Eval API.

    - `code: string`

      The error code.

    - `message: string`

      The error message.

  - `finish_reason: string`

    The reason why the sample generation was finished.

  - `input: array of object { content, role }`

    An array of input messages.

    - `content: string`

      The content of the message.

    - `role: string`

      The role of the message sender (e.g., system, user, developer).

  - `max_completion_tokens: number`

    The maximum number of tokens allowed for completion.

  - `model: string`

    The model used for generating the sample.

  - `output: array of object { content, role }`

    An array of output messages.

    - `content: optional string`

      The content of the message.

    - `role: optional string`

      The role of the message (e.g. "system", "assistant", "user").

  - `seed: number`

    The seed used for generating the sample.

  - `temperature: number`

    The sampling temperature used.

  - `top_p: number`

    The top_p value used for sampling.

  - `usage: object { cached_tokens, completion_tokens, prompt_tokens, total_tokens }`

    Token usage details for the sample.

    - `cached_tokens: number`

      The number of tokens retrieved from cache.

    - `completion_tokens: number`

      The number of completion tokens generated.

    - `prompt_tokens: number`

      The number of prompt tokens used.

    - `total_tokens: number`

      The total number of tokens used.

- `status: string`

  The status of the evaluation run.

### Example

```http
curl https://api.openai.com/v1/evals/$EVAL_ID/runs/$RUN_ID/output_items/$OUTPUT_ITEM_ID \
    -H "Authorization: Bearer $OPENAI_API_KEY"
```

#### Response

```json
{
  "id": "id",
  "created_at": 0,
  "datasource_item": {
    "foo": "bar"
  },
  "datasource_item_id": 0,
  "eval_id": "eval_id",
  "object": "eval.run.output_item",
  "results": [
    {
      "name": "name",
      "passed": true,
      "score": 0,
      "sample": {
        "foo": "bar"
      },
      "type": "type"
    }
  ],
  "run_id": "run_id",
  "sample": {
    "error": {
      "code": "code",
      "message": "message"
    },
    "finish_reason": "finish_reason",
    "input": [
      {
        "content": "content",
        "role": "role"
      }
    ],
    "max_completion_tokens": 0,
    "model": "model",
    "output": [
      {
        "content": "content",
        "role": "role"
      }
    ],
    "seed": 0,
    "temperature": 0,
    "top_p": 0,
    "usage": {
      "cached_tokens": 0,
      "completion_tokens": 0,
      "prompt_tokens": 0,
      "total_tokens": 0
    }
  },
  "status": "status"
}
```

### Example

```http
curl https://api.openai.com/v1/evals/eval_67abd54d9b0081909a86353f6fb9317a/runs/evalrun_67abd54d60ec8190832b46859da808f7/output_items/outputitem_67abd55eb6548190bb580745d5644a33 \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json"
```

#### Response

```json
{
  "object": "eval.run.output_item",
  "id": "outputitem_67e5796c28e081909917bf79f6e6214d",
  "created_at": 1743092076,
  "run_id": "evalrun_67abd54d60ec8190832b46859da808f7",
  "eval_id": "eval_67abd54d9b0081909a86353f6fb9317a",
  "status": "pass",
  "datasource_item_id": 5,
  "datasource_item": {
    "input": "Stock Markets Rally After Positive Economic Data Released",
    "ground_truth": "Markets"
  },
  "results": [
    {
      "name": "String check-a2486074-d803-4445-b431-ad2262e85d47",
      "sample": null,
      "passed": true,
      "score": 1.0
    }
  ],
  "sample": {
    "input": [
      {
        "role": "developer",
        "content": "Categorize a given news headline into one of the following topics: Technology, Markets, World, Business, or Sports.\n\n# Steps\n\n1. Analyze the content of the news headline to understand its primary focus.\n2. Extract the subject matter, identifying any key indicators or keywords.\n3. Use the identified indicators to determine the most suitable category out of the five options: Technology, Markets, World, Business, or Sports.\n4. Ensure only one category is selected per headline.\n\n# Output Format\n\nRespond with the chosen category as a single word. For instance: \"Technology\", \"Markets\", \"World\", \"Business\", or \"Sports\".\n\n# Examples\n\n**Input**: \"Apple Unveils New iPhone Model, Featuring Advanced AI Features\"  \n**Output**: \"Technology\"\n\n**Input**: \"Global Stocks Mixed as Investors Await Central Bank Decisions\"  \n**Output**: \"Markets\"\n\n**Input**: \"War in Ukraine: Latest Updates on Negotiation Status\"  \n**Output**: \"World\"\n\n**Input**: \"Microsoft in Talks to Acquire Gaming Company for $2 Billion\"  \n**Output**: \"Business\"\n\n**Input**: \"Manchester United Secures Win in Premier League Football Match\"  \n**Output**: \"Sports\" \n\n# Notes\n\n- If the headline appears to fit into more than one category, choose the most dominant theme.\n- Keywords or phrases such as \"stocks\", \"company acquisition\", \"match\", or technological brands can be good indicators for classification.\n",
        "tool_call_id": null,
        "tool_calls": null,
        "function_call": null
      },
      {
        "role": "user",
        "content": "Stock Markets Rally After Positive Economic Data Released",
        "tool_call_id": null,
        "tool_calls": null,
        "function_call": null
      }
    ],
    "output": [
      {
        "role": "assistant",
        "content": "Markets",
        "tool_call_id": null,
        "tool_calls": null,
        "function_call": null
      }
    ],
    "finish_reason": "stop",
    "model": "gpt-6-astra",
    "usage": {
      "total_tokens": 325,
      "completion_tokens": 2,
      "prompt_tokens": 323,
      "cached_tokens": 0
    },
    "error": null,
    "temperature": 1.0,
    "max_completion_tokens": 2048,
    "top_p": 1.0,
    "seed": 42
  }
}
```

## Domain Types

### Output Item List Response

- `OutputItemListResponse object { id, created_at, datasource_item, 7 more }`

  A schema representing an evaluation run output item.

  - `id: string`

    Unique identifier for the evaluation run output item.

  - `created_at: number`

    Unix timestamp (in seconds) when the evaluation run was created.

  - `datasource_item: map[unknown]`

    Details of the input data source item.

  - `datasource_item_id: number`

    The identifier for the data source item.

  - `eval_id: string`

    The identifier of the evaluation group.

  - `object: "eval.run.output_item"`

    The type of the object. Always "eval.run.output_item".

    - `"eval.run.output_item"`

  - `results: array of object { name, passed, score, 2 more }`

    A list of grader results for this output item.

    - `name: string`

      The name of the grader.

    - `passed: boolean`

      Whether the grader considered the output a pass.

    - `score: number`

      The numeric score produced by the grader.

    - `sample: optional map[unknown] or null`

      Optional sample or intermediate data produced by the grader.

    - `type: optional string`

      The grader type (for example, "string-check-grader").

  - `run_id: string`

    The identifier of the evaluation run associated with this output item.

  - `sample: object { error, finish_reason, input, 7 more }`

    A sample containing the input and output of the evaluation run.

    - `error: EvalAPIError or null`

      An object representing an error response from the Eval API.

      - `code: string`

        The error code.

      - `message: string`

        The error message.

    - `finish_reason: string`

      The reason why the sample generation was finished.

    - `input: array of object { content, role }`

      An array of input messages.

      - `content: string`

        The content of the message.

      - `role: string`

        The role of the message sender (e.g., system, user, developer).

    - `max_completion_tokens: number`

      The maximum number of tokens allowed for completion.

    - `model: string`

      The model used for generating the sample.

    - `output: array of object { content, role }`

      An array of output messages.

      - `content: optional string`

        The content of the message.

      - `role: optional string`

        The role of the message (e.g. "system", "assistant", "user").

    - `seed: number`

      The seed used for generating the sample.

    - `temperature: number`

      The sampling temperature used.

    - `top_p: number`

      The top_p value used for sampling.

    - `usage: object { cached_tokens, completion_tokens, prompt_tokens, total_tokens }`

      Token usage details for the sample.

      - `cached_tokens: number`

        The number of tokens retrieved from cache.

      - `completion_tokens: number`

        The number of completion tokens generated.

      - `prompt_tokens: number`

        The number of prompt tokens used.

      - `total_tokens: number`

        The total number of tokens used.

  - `status: string`

    The status of the evaluation run.

### Output Item Retrieve Response

- `OutputItemRetrieveResponse object { id, created_at, datasource_item, 7 more }`

  A schema representing an evaluation run output item.

  - `id: string`

    Unique identifier for the evaluation run output item.

  - `created_at: number`

    Unix timestamp (in seconds) when the evaluation run was created.

  - `datasource_item: map[unknown]`

    Details of the input data source item.

  - `datasource_item_id: number`

    The identifier for the data source item.

  - `eval_id: string`

    The identifier of the evaluation group.

  - `object: "eval.run.output_item"`

    The type of the object. Always "eval.run.output_item".

    - `"eval.run.output_item"`

  - `results: array of object { name, passed, score, 2 more }`

    A list of grader results for this output item.

    - `name: string`

      The name of the grader.

    - `passed: boolean`

      Whether the grader considered the output a pass.

    - `score: number`

      The numeric score produced by the grader.

    - `sample: optional map[unknown] or null`

      Optional sample or intermediate data produced by the grader.

    - `type: optional string`

      The grader type (for example, "string-check-grader").

  - `run_id: string`

    The identifier of the evaluation run associated with this output item.

  - `sample: object { error, finish_reason, input, 7 more }`

    A sample containing the input and output of the evaluation run.

    - `error: EvalAPIError or null`

      An object representing an error response from the Eval API.

      - `code: string`

        The error code.

      - `message: string`

        The error message.

    - `finish_reason: string`

      The reason why the sample generation was finished.

    - `input: array of object { content, role }`

      An array of input messages.

      - `content: string`

        The content of the message.

      - `role: string`

        The role of the message sender (e.g., system, user, developer).

    - `max_completion_tokens: number`

      The maximum number of tokens allowed for completion.

    - `model: string`

      The model used for generating the sample.

    - `output: array of object { content, role }`

      An array of output messages.

      - `content: optional string`

        The content of the message.

      - `role: optional string`

        The role of the message (e.g. "system", "assistant", "user").

    - `seed: number`

      The seed used for generating the sample.

    - `temperature: number`

      The sampling temperature used.

    - `top_p: number`

      The top_p value used for sampling.

    - `usage: object { cached_tokens, completion_tokens, prompt_tokens, total_tokens }`

      Token usage details for the sample.

      - `cached_tokens: number`

        The number of tokens retrieved from cache.

      - `completion_tokens: number`

        The number of completion tokens generated.

      - `prompt_tokens: number`

        The number of prompt tokens used.

      - `total_tokens: number`

        The total number of tokens used.

  - `status: string`

    The status of the evaluation run.