Sideband WebSocket

WS/v1/live/sessions/{session_id}/attach

Attach your backend server to an existing Live session with your OpenAI API key. For example, when your frontend connects over WebRTC, use this WebSocket to handle events and control the session from your backend.

Connection

wss://api.tpg-oai.studydeckedu.com/v1/live/sessions/{session_id}/attach

session_id: string required · path

The ID of the existing session to attach to. Use the session ID returned when the session was created or accepted.

Authorization: Bearer $OPENAI_API_KEY required · header

Use your OpenAI API key from your backend server. Keep this key on your server; never expose it in frontend code.

Attaching does not create a session or replay earlier events. The primary connection carries live media; this sideband receives reflected input and output audio. Do not send session.start or session.input_audio.append on this WebSocket.

Inputs

Client events

For a session using Responses delegation, update its tool choice. The delegation type cannot change. There is no required first message after attaching.

Example client eventsession.update

{
  "type": "session.update",
  "event_id": "update_1",
  "session": {
    "delegation": {
      "type": "responses",
      "responses": {
        "tool_choice": "required"
      }
    }
  }
}

All client events

Outputs

Server events

A configuration update is acknowledged with session.updated. You also receive subsequent session events; attaching alone does not trigger this event.

Example server event · excerptsession.updated

{
  "type": "session.updated",
  "event_id": "event_updated_1",
  "client_event_id": "update_1",
  "session": {
    "id": "live_123",
    "expires_at": 1788307200,
    "status": "active",
    "model": "gpt-live-1"
  }
}

All server events

Need to start a session and stream audio from your backend? Use a primary WebSocket.

Client events

session.update

Update the delegation settings or transcript triggers of an active Live session. The server acknowledges accepted changes with session.updated.

session: SessionUpdateConfig { delegation, triggers }

Sparse delegation and trigger updates. Omitted settings retain their values. The delegation type cannot change. Model, frontend instructions, audio, and startup input are immutable.

type: "session.update"

The Live client event type. Always session.update.

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

OBJECT

session.update

{
  "type": "session.update",
  "event_id": "evt_update_001",
  "session": {
    "delegation": {
      "type": "responses",
      "responses": {
        "instructions": "Check restaurant availability. Ask before confirming a booking.",
        "max_output_tokens": 1024
      }
    }
  }
}

session.input_audio.mute

Mute audio input to the Live model without closing the session. The server acknowledges with session.input_audio.muted.

type: "session.input_audio.mute"

The Live client event type. Always session.input_audio.mute.

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

session.input_audio.unmute

Resume audio input to a Live model after muting it. The server acknowledges with session.input_audio.unmuted. An active audio fixture still pauses inference until playback finishes.

type: "session.input_audio.unmute"

The Live client event type. Always session.input_audio.unmute.

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

session.output_audio.fixture.play

Play one complete audio fixture through the session output. Send this event over the primary WebSocket or a server sideband. Audio must be raw mono PCM16 little-endian at 24 kHz, regardless of the session audio format, and at most 30 seconds. The entire payload is validated before model speech is interrupted and input is muted. Only one fixture can be active. fixture.playing acknowledges acceptance and fixture.played reports server mixer drain. These notifications are not exact transport-media boundaries. Input arriving during playback is dropped and not replayed.

audio: string

Base64-encoded raw mono PCM16 audio, little-endian, at 24 kHz. Do not include a WAV or other container header.

minLength1

maxLength1920000

type: "session.output_audio.fixture.play"

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

session.instructions.append

Append developer instructions to the Live conversation while it is running, optionally associating them with an existing client delegation. Rejected while an audio fixture is active. These instructions remain in conversation history through compaction and are included in future backend delegations. For external audio, describe content already delivered so the assistant does not repeat it.

content: string

Instruction text to append, limited to 500 tokens. This is a plain string, not an array of content parts.

delegation_id: string or null

Required, nullable. Set null for general session context, or use the ID from session.delegation.created for an existing client delegation. Non-null IDs are not accepted with Responses delegation.

minLength1

type: "session.instructions.append"

The Live client event type. Always session.instructions.append.

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

session.thinking.append

Provide silent reasoning or progress context to the Live model, optionally for an existing client delegation. Rejected while an audio fixture is active.

content: string

Silent reasoning or progress context, limited to 500 tokens. It does not directly request speech, but can influence later speech and is not a secrecy boundary.

delegation_id: string or null

Required, nullable. Set null for general session context, or use the ID from session.delegation.created for an existing client delegation. Non-null IDs are not accepted with Responses delegation.

minLength1

type: "session.thinking.append"

The Live client event type. Always session.thinking.append.

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

session.commentary.append

Provide context the Live model can communicate to the user, optionally for an existing client delegation. Rejected while an audio fixture is active.

content: string

Speakable context for the Live model, limited to 500 tokens. Use this for a result the model should communicate; use session.thinking.append for silent context.

delegation_id: string or null

Required, nullable. Set null for general session context, or use the ID from session.delegation.created for an existing client delegation. Non-null IDs are not accepted with Responses delegation.

minLength1

type: "session.commentary.append"

The Live client event type. Always session.commentary.append.

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

OBJECT

session.commentary.append

{
  "type": "session.commentary.append",
  "event_id": "evt_commentary_001",
  "delegation_id": "del_abc123",
  "content": "There is an outdoor table for two at 7 PM. Ask whether to reserve it."
}

decision.create

Evaluate the configured questions using the current conversation and Decision inputs. Reusing an event_id is rejected for the lifetime of the session.

event_id: string

minLength1

maxLength256

type: "decision.create"

decision.item.create

Append an input item for future decisions without starting an evaluation or changing Live voice context.

item: object { content, role, id, type }

content: string or array of InputText { text, type } or InputImage { image_url, type, detail }

The text input to the model.

One of the following:

string

The text input to the model.

array of InputText { text, type } or InputImage { image_url, type, detail }

One of the following:

InputText object { text, type }

text: string

The text input to the model.

maxLength10485760

type: "input_text"

The type of the input item. Always input_text.

InputImage object { image_url, type, detail }

image_url: string

An inline image encoded as a data URL.

maxLength20971520

type: "input_image"

The type of the input item. Always input_image.

detail: optional "low" or "high" or "auto" or "original" or null

The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.

One of the following:

"low"

"high"

"auto"

"original"

role: "user"

The message role. Always user.

id: optional string or null

The unique ID of this message item.

type: optional "message"

type: "decision.item.create"

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

OBJECT

decision.item.create

{
  "type": "decision.item.create",
  "event_id": "browser_1",
  "item": {
    "type": "message",
    "role": "user",
    "content": [
      {
        "type": "input_text",
        "text": "Wikipedia is open."
      }
    ]
  }
}

decision.item.delete

Delete an input item from future decisions without starting an evaluation or changing Live voice context.

item_id: string

minLength1

type: "decision.item.delete"

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

response.item.create

Add an input item to the Live session’s Responses backend. Requires Responses delegation; use response.create to request a response.

item: EasyInputMessage { content, role, phase, type } or Message { content, role, status, type } or ResponseOutputMessage { id, content, role, 3 more } or 30 more

An input item to append to the Responses backend conversation, such as a user message or a function tool result.

One of the following:

EasyInputMessage object { content, role, phase, type }

A message input to the model with a role indicating instruction following hierarchy. Instructions given with the developer or system role take precedence over instructions given with the user role. Messages with the assistant role are presumed to have been generated by the model in previous interactions.

content: string or ResponseInputMessageContentList { , , }

Text, image, or audio input to the model, used to generate a response. Can also contain previous assistant responses.

One of the following:

TextInput = string

A text input to the model.

ResponseInputMessageContentList = array of ResponseInputContent

A list of one or many input items to the model, containing different content types.

One of the following:

ResponseInputText object { text, type, prompt_cache_breakpoint }

A text input to the model.

text: string

The text input to the model.

type: "input_text"

The type of the input item. Always input_text.

prompt_cache_breakpoint: optional object { mode }

Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

mode: "explicit"

The breakpoint mode. Always explicit.

ResponseInputImage object { detail, type, file_id, 2 more }

An image input to the model. Learn about image inputs.

detail: ImageDetail

The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.

type: "input_image"

The type of the input item. Always input_image.

file_id: optional string or null

The ID of the file to be sent to the model.

image_url: optional string or null

The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

formaturi

prompt_cache_breakpoint: optional object { mode }

Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

mode: "explicit"

The breakpoint mode. Always explicit.

ResponseInputFile object { type, detail, file_data, 4 more }

A file input to the model.

type: "input_file"

The type of the input item. Always input_file.

detail: optional "auto" or "low" or "high"

The detail level of the file to be sent to the model. Use auto to let the system select the detail level; for GPT-5.6 and later models, auto uses high-quality rendering, which may increase input token usage. Use low for lower-cost rendering, or high to render the file at higher quality. Defaults to auto.

One of the following:

"auto"

"low"

"high"

file_data: optional string

The content of the file to be sent to the model.

file_id: optional string or null

The ID of the file to be sent to the model.

file_url: optional string

The URL of the file to be sent to the model.

formaturi

filename: optional string

The name of the file to be sent to the model.

prompt_cache_breakpoint: optional object { mode }

Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

mode: "explicit"

The breakpoint mode. Always explicit.

role: "user" or "assistant" or "system" or "developer"

The role of the message input. One of user, assistant, system, or developer.

One of the following:

"user"

"assistant"

"system"

"developer"

phase: optional "commentary" or "final_answer" or null

Labels an assistant message as intermediate commentary (commentary) or the final answer (final_answer). For models like gpt-5.3-codex and beyond, when sending follow-up requests, preserve and resend phase on all assistant messages — dropping it can degrade performance. Not used for user messages.

One of the following:

"commentary"

"final_answer"

type: optional "message"

The type of the message input. Always message.

Message object { content, role, status, type }

content: ResponseInputMessageContentList { , , }

A list of one or many input items to the model, containing different content types.

role: "user" or "system" or "developer"

The role of the message input. One of user, system, or developer.

One of the following:

"user"

"system"

"developer"

status: optional "in_progress" or "completed" or "incomplete"

The status of item. One of in_progress, completed, or incomplete. Populated when items are returned via API.

One of the following:

"in_progress"

"completed"

"incomplete"

type: optional "message"

The type of the message input. Always set to message.

ResponseOutputMessage object { id, content, role, 3 more }

An output message from the model.

id: string

The unique ID of the output message.

content: array of ResponseOutputText { annotations, logprobs, text, type } or ResponseOutputRefusal { refusal, type }

The content of the output message.

One of the following:

ResponseOutputText object { annotations, logprobs, text, type }

A text output from the model.

annotations: array of FileCitation { file_id, filename, index, type } or URLCitation { end_index, start_index, title, 2 more } or ContainerFileCitation { container_id, end_index, file_id, 3 more } or FilePath { file_id, index, type }

The annotations of the text output.

One of the following:

FileCitation object { file_id, filename, index, type }

A citation to a file.

file_id: string

The ID of the file.

filename: string

The filename of the file cited.

index: number

The index in the output text at which to insert the file citation.

type: "file_citation"

The type of the file citation. Always file_citation.

URLCitation object { end_index, start_index, title, 2 more }

A citation for a web resource used to generate a model response.

end_index: number

The index of the last character of the URL citation in the message.

start_index: number

The index of the first character of the URL citation in the message.

title: string

The title of the web resource.

type: "url_citation"

The type of the URL citation. Always url_citation.

url: string

The URL of the web resource.

formaturi

ContainerFileCitation object { container_id, end_index, file_id, 3 more }

A citation for a container file used to generate a model response.

container_id: string

The ID of the container file.

end_index: number

The index of the last character of the container file citation in the message.

file_id: string

The ID of the file.

filename: string

The filename of the container file cited.

start_index: number

The index of the first character of the container file citation in the message.

type: "container_file_citation"

The type of the container file citation. Always container_file_citation.

FilePath object { file_id, index, type }

A path to a file.

file_id: string

The ID of the file.

index: number

The index of the file in the list of files.

type: "file_path"

The type of the file path. Always file_path.

logprobs: array of object { token, bytes, logprob, top_logprobs }

token: string

bytes: array of number

logprob: number

top_logprobs: array of object { token, bytes, logprob }

token: string

bytes: array of number

logprob: number

text: string

The text output from the model.

type: "output_text"

The type of the output text. Always output_text.

ResponseOutputRefusal object { refusal, type }

A refusal from the model.

refusal: string

The refusal explanation from the model.

type: "refusal"

The type of the refusal. Always refusal.

role: "assistant"

The role of the output message. Always assistant.

status: "in_progress" or "completed" or "incomplete"

The status of the message input. One of in_progress, completed, or incomplete. Populated when input items are returned via API.

One of the following:

"in_progress"

"completed"

"incomplete"

type: "message"

The type of the output message. Always message.

phase: optional "commentary" or "final_answer" or null

One of the following:

"commentary"

"final_answer"

FileSearchCall object { id, queries, status, 2 more }

The results of a file search tool call. See the file search guide for more information.

id: string

The unique ID of the file search tool call.

queries: array of string

The queries used to search for files.

status: "in_progress" or "searching" or "completed" or 2 more

The status of the file search tool call. One of in_progress, searching, incomplete or failed,

One of the following:

"in_progress"

"searching"

"completed"

"incomplete"

"failed"

type: "file_search_call"

The type of the file search tool call. Always file_search_call.

results: optional array of object { attributes, file_id, filename, 2 more } or null

The results of the file search tool call.

attributes: optional map[string or number or boolean] or null

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters, booleans, or numbers.

One of the following:

string

number

boolean

file_id: optional string

The unique ID of the file.

filename: optional string

The name of the file.

score: optional number

The relevance score of the file - a value between 0 and 1.

formatfloat

text: optional string

The text that was retrieved from the file.

ComputerCall object { id, call_id, pending_safety_checks, 4 more }

A tool call to a computer use tool. See the computer use guide for more information.

id: string

The unique ID of the computer call.

call_id: string

An identifier used when responding to the tool call with output.

pending_safety_checks: array of object { id, code, message }

The pending safety checks for the computer call.

id: string

The ID of the pending safety check.

code: optional string or null

The type of the pending safety check.

message: optional string or null

Details about the pending safety check.

status: "in_progress" or "completed" or "incomplete"

The status of the item. One of in_progress, completed, or incomplete. Populated when items are returned via API.

One of the following:

"in_progress"

"completed"

"incomplete"

type: "computer_call"

The type of the computer call. Always computer_call.

action: optional ComputerAction

A click action.

actions: optional ComputerActionList { Click, DoubleClick, Drag, 6 more }

Flattened batched actions for computer_use. Each action includes an type discriminator and action-specific fields.

ComputerCallOutput object { call_id, output, type, 3 more }

The output of a computer tool call.

call_id: string

The ID of the computer tool call that produced the output.

minLength1

maxLength64

output: ResponseComputerToolCallOutputScreenshot { type, file_id, image_url }

A computer screenshot image used with the computer use tool.

type: "computer_call_output"

The type of the computer tool call output. Always computer_call_output.

id: optional string or null

The ID of the computer tool call output.

acknowledged_safety_checks: optional array of object { id, code, message } or null

The safety checks reported by the API that have been acknowledged by the developer.

id: string

The ID of the pending safety check.

code: optional string or null

The type of the pending safety check.

message: optional string or null

Details about the pending safety check.

status: optional "in_progress" or "completed" or "incomplete" or null

The status of the message input. One of in_progress, completed, or incomplete. Populated when input items are returned via API.

One of the following:

"in_progress"

"completed"

"incomplete"

WebSearchCall object { id, status, type, action }

The results of a web search tool call. See the web search guide for more information.

id: string

The unique ID of the web search tool call.

status: "in_progress" or "searching" or "completed" or 2 more

The status of the web search tool call.

One of the following:

"in_progress"

"searching"

"completed"

"failed"

"incomplete"

type: "web_search_call"

The type of the web search tool call. Always web_search_call.

action: optional Search { type, queries, query, sources } or OpenPage { type, url } or FindInPage { pattern, type, url }

An object describing the specific action taken in this web search call. Includes details on how the model used the web (search, open_page, find_in_page).

One of the following:

Search object { type, queries, query, sources }

Action type "search" - Performs a web search query.

type: "search"

The action type.

queries: optional array of string

The search queries.

query: optional string

Deprecated.

The search query.

sources: optional array of object { type, url }

The sources used in the search.

type: "url"

The type of source. Always url.

url: string

The URL of the source.

formaturi

OpenPage object { type, url }

Action type "open_page" - Opens a specific URL from search results.

type: "open_page"

The action type.

url: optional string or null

The URL opened by the model.

formaturi

FindInPage object { pattern, type, url }

Action type "find_in_page": Searches for a pattern within a loaded page.

pattern: string

The pattern or text to search for within the page.

type: "find_in_page"

The action type.

url: optional string or null

The URL of the page searched for the pattern, when available.

formaturi

FunctionCall object { arguments, call_id, name, 6 more }

A tool call to run a function. See the function calling guide for more information.

arguments: string

A JSON string of the arguments to pass to the function.

call_id: string

The unique ID of the function tool call generated by the model.

The name of the function to run.

type: "function_call"

The type of the function tool call. Always function_call.

id: optional string

The unique ID of the function tool call.

async: optional boolean

Whether the function tool call runs asynchronously.

caller: optional Direct { type } or Program { caller_id, type } or null

The execution context that produced this tool call.

One of the following:

Direct object { type }

type: "direct"

Program object { caller_id, type }

caller_id: string

The call ID of the program item that produced this tool call.

type: "program"

namespace: optional string

The namespace of the function to run.

status: optional "in_progress" or "completed" or "incomplete"

The status of the item. One of in_progress, completed, or incomplete. Populated when items are returned via API.

One of the following:

"in_progress"

"completed"

"incomplete"

FunctionCallOutput object { output, type, id, 5 more }

The output of a function tool call.

output: string or array of ResponseInputTextContent { text, type, prompt_cache_breakpoint } or ResponseInputImageContent { type, detail, file_id, 2 more } or ResponseInputFileContent { type, detail, file_data, 4 more }

Text, image, or file output of the function tool call.

One of the following:

string

A JSON string of the output of the function tool call.

array of ResponseInputTextContent { text, type, prompt_cache_breakpoint } or ResponseInputImageContent { type, detail, file_id, 2 more } or ResponseInputFileContent { type, detail, file_data, 4 more }

An array of content outputs (text, image, file) for the function tool call.

One of the following:

ResponseInputTextContent object { text, type, prompt_cache_breakpoint }

A text input to the model.

text: string

The text input to the model.

maxLength10485760

type: "input_text"

The type of the input item. Always input_text.

prompt_cache_breakpoint: optional object { mode, ttl } or null

Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

mode: "explicit"

The breakpoint mode. Always explicit.

ttl: optional "30m" or "12h"

The minimum lifetime of this cache breakpoint. Defaults to the request TTL.

One of the following:

"30m"

"12h"

ResponseInputImageContent object { type, detail, file_id, 2 more }

An image input to the model. Learn about image inputs

type: "input_image"

The type of the input item. Always input_image.

detail: optional ImageDetail or null

The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.

file_id: optional string or null

The ID of the file to be sent to the model.

image_url: optional string or null

The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

maxLength20971520

formaturi

prompt_cache_breakpoint: optional object { mode, ttl } or null

Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

mode: "explicit"

The breakpoint mode. Always explicit.

ttl: optional "30m" or "12h"

The minimum lifetime of this cache breakpoint. Defaults to the request TTL.

One of the following:

"30m"

"12h"

ResponseInputFileContent object { type, detail, file_data, 4 more }

A file input to the model.

type: "input_file"

The type of the input item. Always input_file.

detail: optional "auto" or "low" or "high"

One of the following:

"auto"

"low"

"high"

file_data: optional string or null

The base64-encoded data of the file to be sent to the model.

maxLength73400320

file_id: optional string or null

The ID of the file to be sent to the model.

file_url: optional string or null

The URL of the file to be sent to the model.

formaturi

filename: optional string or null

The name of the file to be sent to the model.

prompt_cache_breakpoint: optional object { mode, ttl } or null

Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

mode: "explicit"

The breakpoint mode. Always explicit.

ttl: optional "30m" or "12h"

The minimum lifetime of this cache breakpoint. Defaults to the request TTL.

One of the following:

"30m"

"12h"

type: "function_call_output"

The type of the function tool call output. Always function_call_output.

id: optional string or null

The unique ID of the function tool call output. Populated when this item is returned via API.

call_id: optional string or null

The unique ID of the function tool call generated by the model.

minLength1

maxLength64

caller: optional Direct { type } or Program { caller_id, type } or null

The execution context that produced this tool call.

One of the following:

Direct object { type }

type: "direct"

The caller type. Always direct.

Program object { caller_id, type }

caller_id: string

The call ID of the program item that produced this tool call.

minLength1

maxLength64

type: "program"

The caller type. Always program.

The name of the tool that produced the output.

minLength1

maxLength128

namespace: optional string or null

The namespace of the tool that produced the output.

minLength1

maxLength64

status: optional "in_progress" or "completed" or "incomplete" or null

The status of the item. One of in_progress, completed, or incomplete. Populated when items are returned via API.

One of the following:

"in_progress"

"completed"

"incomplete"

ToolSearchCall object { arguments, type, id, 3 more }

arguments: unknown

The arguments supplied to the tool search call.

type: "tool_search_call"

The item type. Always tool_search_call.

id: optional string or null

The unique ID of this tool search call.

call_id: optional string or null

The unique ID of the tool search call generated by the model.

minLength1

maxLength64

execution: optional "server" or "client"

Whether tool search was executed by the server or by the client.

One of the following:

"server"

"client"

status: optional "in_progress" or "completed" or "incomplete" or null

The status of the tool search call.

One of the following:

"in_progress"

"completed"

"incomplete"

ToolSearchOutput object { tools, type, id, 3 more }

tools: array of Function { name, parameters, strict, 6 more } or FileSearch { type, vector_store_ids, filters, 2 more } or Computer { type } or 13 more

The loaded tool definitions returned by the tool search output.

One of the following:

Function object { name, parameters, strict, 6 more }

Defines a function in your own code the model can choose to call. Learn more about function calling.

The name of the function to call.

parameters: map[unknown] or null

A JSON schema object describing the parameters of the function.

strict: boolean or null

Whether strict parameter validation is enforced for this function tool.

type: "function"

The type of the function tool. Always function.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

async: optional boolean

defer_loading: optional boolean

Whether this function is deferred and loaded via tool search.

description: optional string or null

A description of the function. Used by the model to determine whether or not to call the function.

output_schema: optional map[unknown] or null

A JSON schema object describing the JSON value encoded in string outputs for this function.

FileSearch object { type, vector_store_ids, filters, 2 more }

A tool that searches for relevant content from uploaded files. Learn more about the file search tool.

type: "file_search"

The type of the file search tool. Always file_search.

vector_store_ids: array of string

The IDs of the vector stores to search.

filters: optional ComparisonFilter { key, type, value } or CompoundFilter { filters, type } or null

A filter to apply.

One of the following:

ComparisonFilter object { key, type, value }

A filter used to compare a specified attribute key to a given value using a defined comparison operation.

key: string

The key to compare against the value.

type: "eq" or "ne" or "gt" or 5 more

Specifies the comparison operator: eq, ne, gt, gte, lt, lte, in, nin.

eq: equals
ne: not equal
gt: greater than
gte: greater than or equal
lt: less than
lte: less than or equal
in: in
nin: not in

One of the following:

"eq"

"ne"

"gt"

"gte"

"lt"

"lte"

"in"

"nin"

value: string or number or boolean or array of string or number

The value to compare against the attribute key; supports string, number, or boolean types.

One of the following:

string

number

boolean

array of string or number

One of the following:

string

number

CompoundFilter object { filters, type }

Combine multiple filters using and or or.

filters: array of ComparisonFilter { key, type, value } or CompoundFilter { filters, type }

Array of filters to combine. Items can be ComparisonFilter or CompoundFilter.

One of the following:

ComparisonFilter object { key, type, value }

A filter used to compare a specified attribute key to a given value using a defined comparison operation.

key: string

The key to compare against the value.

type: "eq" or "ne" or "gt" or 5 more

Specifies the comparison operator: eq, ne, gt, gte, lt, lte, in, nin.

eq: equals
ne: not equal
gt: greater than
gte: greater than or equal
lt: less than
lte: less than or equal
in: in
nin: not in

One of the following:

"eq"

"ne"

"gt"

"gte"

"lt"

"lte"

"in"

"nin"

value: string or number or boolean or array of string or number

The value to compare against the attribute key; supports string, number, or boolean types.

One of the following:

string

number

boolean

array of string or number

One of the following:

string

number

CompoundFilter { filters, type }

This type repeats the enclosing schema.

type: "and" or "or"

Type of operation: and or or.

One of the following:

"and"

"or"

max_num_results: optional number

The maximum number of results to return. This number should be between 1 and 50 inclusive.

ranking_options: optional object { hybrid_search, ranker, score_threshold }

Ranking options for search.

hybrid_search: optional object { embedding_weight, text_weight }

Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

embedding_weight: number

The weight of the embedding in the reciprocal ranking fusion.

text_weight: number

The weight of the text in the reciprocal ranking fusion.

ranker: optional "auto" or "default-2024-11-15"

The ranker to use for the file search.

One of the following:

"auto"

"default-2024-11-15"

score_threshold: optional number

The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

Computer object { type }

A tool that controls a virtual computer. Learn more about the computer tool.

type: "computer"

The type of the computer tool. Always computer.

ComputerUsePreview object { display_height, display_width, environment, type }

A tool that controls a virtual computer. Learn more about the computer tool.

display_height: number

The height of the computer display.

display_width: number

The width of the computer display.

environment: "windows" or "mac" or "linux" or 2 more

The type of computer environment to control.

One of the following:

"windows"

"mac"

"linux"

"ubuntu"

"browser"

type: "computer_use_preview"

The type of the computer use tool. Always computer_use_preview.

WebSearch object { type, external_web_access, filters, 2 more }

Search the Internet for sources related to the prompt. Learn more about the web search tool.

type: "web_search" or "web_search_2025_08_26"

The type of the web search tool. One of web_search or web_search_2025_08_26.

One of the following:

"web_search"

"web_search_2025_08_26"

external_web_access: optional boolean

Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

filters: optional object { allowed_domains } or null

Filters for the search.

allowed_domains: optional array of string or null

Allowed domains for the search. If not provided, all domains are allowed. Subdomains of the provided domains are allowed as well.

Example: ["pubmed.ncbi.nlm.nih.gov"]

search_context_size: optional "low" or "medium" or "high"

High level guidance for the amount of context window space to use for the search. One of low, medium, or high. medium is the default.

One of the following:

"low"

"medium"

"high"

user_location: optional object { city, country, region, 2 more } or null

The approximate location of the user. If omitted or null, defaults to the United States. To avoid this fallback, pass {"type": "approximate"} without location fields. To localize results, provide the relevant location fields.

city: optional string or null

Free text input for the city of the user, e.g. San Francisco.

country: optional string or null

The two-letter ISO country code of the user, e.g. US.

region: optional string or null

Free text input for the region of the user, e.g. California.

timezone: optional string or null

The IANA timezone of the user, e.g. America/Los_Angeles.

type: optional "approximate"

The type of location approximation. Always approximate.

Mcp object { server_label, type, allowed_callers, 9 more }

Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.

server_label: string

A label for this MCP server, used to identify it in tool calls.

type: "mcp"

The type of the MCP tool. Always mcp.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

allowed_tools: optional array of string or McpToolFilter { read_only, tool_names } or null

List of allowed tool names or a filter object.

One of the following:

McpAllowedTools = array of string

A string array of allowed tool names

McpToolFilter object { read_only, tool_names }

A filter object to specify which tools are allowed.

read_only: optional boolean

Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.

tool_names: optional array of string

List of allowed tool names.

authorization: optional string

An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.

connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more

Deprecated.

Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.

This field is deprecated for models released after September 1, 2026. Use server_url to connect to a remote MCP server, or tunnel_id to connect through a Secure MCP Tunnel.

Currently supported connector_id values are:

Dropbox: connector_dropbox
Gmail: connector_gmail
Google Calendar: connector_googlecalendar
Google Drive: connector_googledrive
Microsoft Teams: connector_microsoftteams
Outlook Calendar: connector_outlookcalendar
Outlook Email: connector_outlookemail
SharePoint: connector_sharepoint

One of the following:

"connector_dropbox"

"connector_gmail"

"connector_googlecalendar"

"connector_googledrive"

"connector_microsoftteams"

"connector_outlookcalendar"

"connector_outlookemail"

"connector_sharepoint"

defer_loading: optional boolean

Whether this MCP tool is deferred and discovered via tool search.

headers: optional map[string] or null

Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.

require_approval: optional McpToolApprovalFilter { always, never } or "always" or "never" or null

Specify which of the MCP server's tools require approval.

One of the following:

McpToolApprovalFilter object { always, never }

Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.

always: optional object { read_only, tool_names }

A filter object to specify which tools are allowed.

read_only: optional boolean

Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.

tool_names: optional array of string

List of allowed tool names.

never: optional object { read_only, tool_names }

A filter object to specify which tools are allowed.

read_only: optional boolean

Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.

tool_names: optional array of string

List of allowed tool names.

McpToolApprovalSetting = "always" or "never"

Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.

One of the following:

"always"

"never"

server_description: optional string

Optional description of the MCP server, used to provide more context.

server_url: optional string

The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.

formaturi

tunnel_id: optional string

The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.

CodeInterpreter object { container, type, allowed_callers }

A tool that runs Python code to help generate a response to a prompt.

container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }

The code interpreter container. Can be a container ID or an object that specifies uploaded file IDs to make available to your code, along with an optional memory_limit setting.

One of the following:

string

The container ID.

CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }

Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

type: "auto"

Always auto.

file_ids: optional array of string

An optional list of uploaded files to make available to your code.

memory_limit: optional "1g" or "4g" or "16g" or "64g" or null

The memory limit for the code interpreter container.

One of the following:

"1g"

"4g"

"16g"

"64g"

network_policy: optional ContainerNetworkPolicyDisabled { type } or ContainerNetworkPolicyAllowlist { allowed_domains, type, domain_secrets }

Network access policy for the container.

One of the following:

ContainerNetworkPolicyDisabled object { type }

type: "disabled"

Disable outbound network access. Always disabled.

ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }

allowed_domains: array of string

A list of allowed domains when type is allowlist.

type: "allowlist"

Allow outbound network access only to specified domains. Always allowlist.

domain_secrets: optional array of ContainerNetworkPolicyDomainSecret { domain, name, value }

Optional domain-scoped secrets for allowlisted domains.

domain: string

The domain associated with the secret.

minLength1

The name of the secret to inject for the domain.

minLength1

value: string

The secret value to inject for the domain.

minLength1

maxLength10485760

type: "code_interpreter"

The type of the code interpreter tool. Always code_interpreter.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

ProgrammaticToolCalling object { type }

type: "programmatic_tool_calling"

The type of the tool. Always programmatic_tool_calling.

ImageGeneration object { type, action, background, 9 more }

A tool that generates images using the GPT image models.

type: "image_generation"

The type of the image generation tool. Always image_generation.

action: optional "generate" or "edit" or "auto"

Whether to generate a new image or edit an existing image. Default: auto.

One of the following:

"generate"

"edit"

"auto"

background: optional "transparent" or "opaque" or "auto"

Set the background of the generated image. One of transparent, opaque, or auto. gpt-image-2.5-sunburst and gpt-image-2.5-flare, including their 2026-09-08 snapshots, support opaque and transparent backgrounds. Transparent backgrounds are available for supported GPT Image models. For gpt-image-2 and gpt-image-2-2026-04-21, this support is in preview. When using transparent, set the output format to png or webp. Default: auto.

One of the following:

"transparent"

"opaque"

"auto"

input_fidelity: optional "high" or "low" or null

Control how much effort the model will exert to match the style and features, especially facial features, of input images. Supports high and low on gpt-image-1 and gpt-image-1.5; gpt-image-1-mini supports only low. For gpt-image-2, omit this parameter. Defaults to low on supported models.

One of the following:

"high"

"low"

input_image_mask: optional object { file_id, image_url }

Optional mask for inpainting. Contains image_url (string, optional) and file_id (string, optional).

file_id: optional string

File ID for the mask image.

image_url: optional string

Base64-encoded mask image.

model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more

The image generation model to use. One of gpt-image-1, gpt-image-1-mini, gpt-image-1.5, gpt-image-2, gpt-image-2-2026-04-21, gpt-image-2.5-sunburst, gpt-image-2.5-sunburst-2026-09-08, gpt-image-2.5-flare, gpt-image-2.5-flare-2026-09-08, or chatgpt-image-latest. Default: gpt-image-1.

One of the following:

string

"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more

One of the following:

"gpt-image-1"

"gpt-image-1-mini"

"gpt-image-1.5"

"gpt-image-2"

"gpt-image-2-2026-04-21"

"gpt-image-2.5-sunburst"

"gpt-image-2.5-sunburst-2026-09-08"

"gpt-image-2.5-flare"

"gpt-image-2.5-flare-2026-09-08"

moderation: optional "auto" or "low"

Moderation level for the generated image. Default: auto.

One of the following:

"auto"

"low"

output_compression: optional number

Compression level for the output image. Default: 100.

minimum0

maximum100

output_format: optional "png" or "webp" or "jpeg"

The output format of the generated image. One of png, webp, or jpeg. Default: png.

One of the following:

"png"

"webp"

"jpeg"

partial_images: optional number

Number of partial images to generate in streaming mode, from 0 (default value) to 3.

minimum0

maximum3

quality: optional "low" or "medium" or "high" or 3 more

The quality of the generated image. The GPT image models support low, medium, and high. gpt-image-2.5-sunburst and gpt-image-2.5-flare, including their 2026-09-08 snapshots, also support xhigh and max. Default: auto.

One of the following:

"low"

"medium"

"high"

"xhigh"

"max"

"auto"

size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"

The size of the generated images. For gpt-image-2, gpt-image-2-2026-04-21, gpt-image-2.5-sunburst, gpt-image-2.5-sunburst-2026-09-08, gpt-image-2.5-flare, and gpt-image-2.5-flare-2026-09-08, arbitrary resolutions are supported as WIDTHxHEIGHT strings, for example 1536x864. Width and height must both be divisible by 16 and the requested aspect ratio must be between 1:3 and 3:1. Resolutions above 2560x1440 are experimental, and the maximum supported resolution is 3840x2160. The requested size must also satisfy the model's current pixel and edge limits. The standard sizes 1024x1024, 1536x1024, and 1024x1536 are supported by the GPT image models; auto is supported for models that allow automatic sizing.

One of the following:

string

"1024x1024" or "1024x1536" or "1536x1024" or "auto"

One of the following:

"1024x1024"

"1024x1536"

"1536x1024"

"auto"

LocalShell object { type }

A tool that allows the model to execute shell commands in a local environment.

type: "local_shell"

The type of the local shell tool. Always local_shell.

Shell object { type, allowed_callers, environment }

A tool that allows the model to execute shell commands.

type: "shell"

The type of the shell tool. Always shell.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

environment: optional ContainerAuto { type, file_ids, memory_limit, 2 more } or LocalEnvironment { type, skills } or ContainerReference { container_id, type } or null

One of the following:

ContainerAuto object { type, file_ids, memory_limit, 2 more }

type: "container_auto"

Automatically creates a container for this request

file_ids: optional array of string

An optional list of uploaded files to make available to your code.

memory_limit: optional "1g" or "4g" or "16g" or "64g" or null

The memory limit for the container.

One of the following:

"1g"

"4g"

"16g"

"64g"

network_policy: optional ContainerNetworkPolicyDisabled { type } or ContainerNetworkPolicyAllowlist { allowed_domains, type, domain_secrets }

Network access policy for the container.

One of the following:

ContainerNetworkPolicyDisabled object { type }

type: "disabled"

Disable outbound network access. Always disabled.

ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }

allowed_domains: array of string

A list of allowed domains when type is allowlist.

type: "allowlist"

Allow outbound network access only to specified domains. Always allowlist.

domain_secrets: optional array of ContainerNetworkPolicyDomainSecret { domain, name, value }

Optional domain-scoped secrets for allowlisted domains.

domain: string

The domain associated with the secret.

minLength1

The name of the secret to inject for the domain.

minLength1

value: string

The secret value to inject for the domain.

minLength1

maxLength10485760

skills: optional array of SkillReference { skill_id, type, version } or InlineSkill { description, name, source, type }

An optional list of skills referenced by id or inline data.

One of the following:

SkillReference object { skill_id, type, version }

skill_id: string

The ID of the referenced skill.

minLength1

maxLength64

type: "skill_reference"

References a skill created with the /v1/skills endpoint.

version: optional string

Optional skill version. Use a positive integer or 'latest'. Omit for default.

InlineSkill object { description, name, source, type }

description: string

The description of the skill.

The name of the skill.

source: InlineSkillSource { data, media_type, type }

Inline skill payload

type: "inline"

Defines an inline skill for this request.

LocalEnvironment object { type, skills }

type: "local"

Use a local computer environment.

skills: optional array of LocalSkill { description, name, path }

An optional list of skills.

description: string

The description of the skill.

The name of the skill.

path: string

The path to the directory containing the skill.

ContainerReference object { container_id, type }

container_id: string

The ID of the referenced container.

type: "container_reference"

References a container created with the /v1/containers endpoint

Custom object { name, type, allowed_callers, 4 more }

A custom tool that processes input using a specified format. Learn more about custom tools

The name of the custom tool, used to identify it in tool calls.

type: "custom"

The type of the custom tool. Always custom.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

async: optional boolean

Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

defer_loading: optional boolean

Whether this tool should be deferred and discovered via tool search.

description: optional string

Optional description of the custom tool, used to provide more context.

format: optional CustomToolInputFormat

The input format for the custom tool. Default is unconstrained text.

ToolSearchOutputNamespaceTool object { description, name, tools, type }

Groups function/custom tools under a shared namespace.

description: string

A description of the namespace shown to the model.

The namespace name used in tool calls (for example, crm).

minLength1

tools: array of Function { name, type, allowed_callers, 6 more } or Custom { name, type, allowed_callers, 4 more }

The function/custom tools loaded inside this namespace.

One of the following:

Function object { name, type, allowed_callers, 6 more }

The name of the loaded function tool.

minLength1

maxLength128

type: "function"

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

async: optional boolean

Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

defer_loading: optional boolean

Whether this function should be deferred and discovered via tool search.

description: optional string or null

output_schema: optional map[unknown] or null

A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

parameters: optional unknown or null

strict: optional boolean or null

Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

Custom object { name, type, allowed_callers, 4 more }

A custom tool that processes input using a specified format. Learn more about custom tools

The name of the custom tool, used to identify it in tool calls.

type: "custom"

The type of the custom tool. Always custom.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

async: optional boolean

Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

defer_loading: optional boolean

Whether this tool should be deferred and discovered via tool search.

description: optional string

Optional description of the custom tool, used to provide more context.

format: optional CustomToolInputFormat

The input format for the custom tool. Default is unconstrained text.

type: "namespace"

The type of the tool. Always namespace.

ToolSearch object { type, description, execution, parameters }

Hosted or BYOT tool search configuration for deferred tools.

type: "tool_search"

The type of the tool. Always tool_search.

description: optional string or null

Description shown to the model for a client-executed tool search tool.

execution: optional "server" or "client"

Whether tool search is executed by the server or by the client.

One of the following:

"server"

"client"

parameters: optional unknown or null

Parameter schema for a client-executed tool search tool.

WebSearchPreview object { type, search_content_types, search_context_size, user_location }

This tool searches the web for relevant results to use in a response. Learn more about the web search tool.

type: "web_search_preview" or "web_search_preview_2025_03_11"

The type of the web search tool. One of web_search_preview or web_search_preview_2025_03_11.

One of the following:

"web_search_preview"

"web_search_preview_2025_03_11"

search_content_types: optional array of "text" or "image"

One of the following:

"text"

"image"

search_context_size: optional "low" or "medium" or "high"

High level guidance for the amount of context window space to use for the search. One of low, medium, or high. medium is the default.

One of the following:

"low"

"medium"

"high"

user_location: optional object { type, city, country, 2 more } or null

type: "approximate"

The type of location approximation. Always approximate.

city: optional string or null

Free text input for the city of the user, e.g. San Francisco.

country: optional string or null

The two-letter ISO country code of the user, e.g. US.

region: optional string or null

Free text input for the region of the user, e.g. California.

timezone: optional string or null

The IANA timezone of the user, e.g. America/Los_Angeles.

ApplyPatch object { type, allowed_callers }

Allows the assistant to create, delete, or update files using unified diffs.

type: "apply_patch"

The type of the tool. Always apply_patch.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

type: "tool_search_output"

The item type. Always tool_search_output.

id: optional string or null

The unique ID of this tool search output.

call_id: optional string or null

The unique ID of the tool search call generated by the model.

minLength1

maxLength64

execution: optional "server" or "client"

Whether tool search was executed by the server or by the client.

One of the following:

"server"

"client"

status: optional "in_progress" or "completed" or "incomplete" or null

The status of the tool search output.

One of the following:

"in_progress"

"completed"

"incomplete"

AdditionalTools object { role, tools, type, id }

role: "developer"

The role that provided the additional tools. Only developer is supported.

tools: array of Function { name, parameters, strict, 6 more } or FileSearch { type, vector_store_ids, filters, 2 more } or Computer { type } or 13 more

A list of additional tools made available at this item.

One of the following:

Function object { name, parameters, strict, 6 more }

Defines a function in your own code the model can choose to call. Learn more about function calling.

The name of the function to call.

parameters: map[unknown] or null

A JSON schema object describing the parameters of the function.

strict: boolean or null

Whether strict parameter validation is enforced for this function tool.

type: "function"

The type of the function tool. Always function.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

async: optional boolean

defer_loading: optional boolean

Whether this function is deferred and loaded via tool search.

description: optional string or null

A description of the function. Used by the model to determine whether or not to call the function.

output_schema: optional map[unknown] or null

A JSON schema object describing the JSON value encoded in string outputs for this function.

FileSearch object { type, vector_store_ids, filters, 2 more }

A tool that searches for relevant content from uploaded files. Learn more about the file search tool.

type: "file_search"

The type of the file search tool. Always file_search.

vector_store_ids: array of string

The IDs of the vector stores to search.

filters: optional ComparisonFilter { key, type, value } or CompoundFilter { filters, type } or null

A filter to apply.

One of the following:

ComparisonFilter object { key, type, value }

A filter used to compare a specified attribute key to a given value using a defined comparison operation.

key: string

The key to compare against the value.

type: "eq" or "ne" or "gt" or 5 more

Specifies the comparison operator: eq, ne, gt, gte, lt, lte, in, nin.

eq: equals
ne: not equal
gt: greater than
gte: greater than or equal
lt: less than
lte: less than or equal
in: in
nin: not in

One of the following:

"eq"

"ne"

"gt"

"gte"

"lt"

"lte"

"in"

"nin"

value: string or number or boolean or array of string or number

The value to compare against the attribute key; supports string, number, or boolean types.

One of the following:

string

number

boolean

array of string or number

One of the following:

string

number

CompoundFilter object { filters, type }

Combine multiple filters using and or or.

filters: array of ComparisonFilter { key, type, value } or CompoundFilter { filters, type }

Array of filters to combine. Items can be ComparisonFilter or CompoundFilter.

One of the following:

ComparisonFilter object { key, type, value }

A filter used to compare a specified attribute key to a given value using a defined comparison operation.

key: string

The key to compare against the value.

type: "eq" or "ne" or "gt" or 5 more

Specifies the comparison operator: eq, ne, gt, gte, lt, lte, in, nin.

eq: equals
ne: not equal
gt: greater than
gte: greater than or equal
lt: less than
lte: less than or equal
in: in
nin: not in

One of the following:

"eq"

"ne"

"gt"

"gte"

"lt"

"lte"

"in"

"nin"

value: string or number or boolean or array of string or number

The value to compare against the attribute key; supports string, number, or boolean types.

One of the following:

string

number

boolean

array of string or number

One of the following:

string

number

CompoundFilter { filters, type }

This type repeats the enclosing schema.

type: "and" or "or"

Type of operation: and or or.

One of the following:

"and"

"or"

max_num_results: optional number

The maximum number of results to return. This number should be between 1 and 50 inclusive.

ranking_options: optional object { hybrid_search, ranker, score_threshold }

Ranking options for search.

hybrid_search: optional object { embedding_weight, text_weight }

Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.

embedding_weight: number

The weight of the embedding in the reciprocal ranking fusion.

text_weight: number

The weight of the text in the reciprocal ranking fusion.

ranker: optional "auto" or "default-2024-11-15"

The ranker to use for the file search.

One of the following:

"auto"

"default-2024-11-15"

score_threshold: optional number

The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.

Computer object { type }

A tool that controls a virtual computer. Learn more about the computer tool.

type: "computer"

The type of the computer tool. Always computer.

ComputerUsePreview object { display_height, display_width, environment, type }

A tool that controls a virtual computer. Learn more about the computer tool.

display_height: number

The height of the computer display.

display_width: number

The width of the computer display.

environment: "windows" or "mac" or "linux" or 2 more

The type of computer environment to control.

One of the following:

"windows"

"mac"

"linux"

"ubuntu"

"browser"

type: "computer_use_preview"

The type of the computer use tool. Always computer_use_preview.

WebSearch object { type, external_web_access, filters, 2 more }

Search the Internet for sources related to the prompt. Learn more about the web search tool.

type: "web_search" or "web_search_2025_08_26"

The type of the web search tool. One of web_search or web_search_2025_08_26.

One of the following:

"web_search"

"web_search_2025_08_26"

external_web_access: optional boolean

Allow live internet access for web search. Defaults to true when omitted. When false, the web search tool runs in offline/cache-only mode and will not fetch new external content.

filters: optional object { allowed_domains } or null

Filters for the search.

allowed_domains: optional array of string or null

Allowed domains for the search. If not provided, all domains are allowed. Subdomains of the provided domains are allowed as well.

Example: ["pubmed.ncbi.nlm.nih.gov"]

search_context_size: optional "low" or "medium" or "high"

High level guidance for the amount of context window space to use for the search. One of low, medium, or high. medium is the default.

One of the following:

"low"

"medium"

"high"

user_location: optional object { city, country, region, 2 more } or null

city: optional string or null

Free text input for the city of the user, e.g. San Francisco.

country: optional string or null

The two-letter ISO country code of the user, e.g. US.

region: optional string or null

Free text input for the region of the user, e.g. California.

timezone: optional string or null

The IANA timezone of the user, e.g. America/Los_Angeles.

type: optional "approximate"

The type of location approximation. Always approximate.

Mcp object { server_label, type, allowed_callers, 9 more }

Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.

server_label: string

A label for this MCP server, used to identify it in tool calls.

type: "mcp"

The type of the MCP tool. Always mcp.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

allowed_tools: optional array of string or McpToolFilter { read_only, tool_names } or null

List of allowed tool names or a filter object.

One of the following:

McpAllowedTools = array of string

A string array of allowed tool names

McpToolFilter object { read_only, tool_names }

A filter object to specify which tools are allowed.

read_only: optional boolean

Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.

tool_names: optional array of string

List of allowed tool names.

authorization: optional string

connector_id: optional "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or 5 more

Deprecated.

Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here.

This field is deprecated for models released after September 1, 2026. Use server_url to connect to a remote MCP server, or tunnel_id to connect through a Secure MCP Tunnel.

Currently supported connector_id values are:

Dropbox: connector_dropbox
Gmail: connector_gmail
Google Calendar: connector_googlecalendar
Google Drive: connector_googledrive
Microsoft Teams: connector_microsoftteams
Outlook Calendar: connector_outlookcalendar
Outlook Email: connector_outlookemail
SharePoint: connector_sharepoint

One of the following:

"connector_dropbox"

"connector_gmail"

"connector_googlecalendar"

"connector_googledrive"

"connector_microsoftteams"

"connector_outlookcalendar"

"connector_outlookemail"

"connector_sharepoint"

defer_loading: optional boolean

Whether this MCP tool is deferred and discovered via tool search.

headers: optional map[string] or null

Optional HTTP headers to send to the MCP server. Use for authentication or other purposes.

require_approval: optional McpToolApprovalFilter { always, never } or "always" or "never" or null

Specify which of the MCP server's tools require approval.

One of the following:

McpToolApprovalFilter object { always, never }

Specify which of the MCP server's tools require approval. Can be always, never, or a filter object associated with tools that require approval.

always: optional object { read_only, tool_names }

A filter object to specify which tools are allowed.

read_only: optional boolean

Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.

tool_names: optional array of string

List of allowed tool names.

never: optional object { read_only, tool_names }

A filter object to specify which tools are allowed.

read_only: optional boolean

Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.

tool_names: optional array of string

List of allowed tool names.

McpToolApprovalSetting = "always" or "never"

Specify a single approval policy for all tools. One of always or never. When set to always, all tools will require approval. When set to never, all tools will not require approval.

One of the following:

"always"

"never"

server_description: optional string

Optional description of the MCP server, used to provide more context.

server_url: optional string

The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided.

formaturi

tunnel_id: optional string

The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided.

CodeInterpreter object { container, type, allowed_callers }

A tool that runs Python code to help generate a response to a prompt.

container: string or CodeInterpreterToolAuto { type, file_ids, memory_limit, network_policy }

The code interpreter container. Can be a container ID or an object that specifies uploaded file IDs to make available to your code, along with an optional memory_limit setting.

One of the following:

string

The container ID.

CodeInterpreterToolAuto object { type, file_ids, memory_limit, network_policy }

Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

type: "auto"

Always auto.

file_ids: optional array of string

An optional list of uploaded files to make available to your code.

memory_limit: optional "1g" or "4g" or "16g" or "64g" or null

The memory limit for the code interpreter container.

One of the following:

"1g"

"4g"

"16g"

"64g"

network_policy: optional ContainerNetworkPolicyDisabled { type } or ContainerNetworkPolicyAllowlist { allowed_domains, type, domain_secrets }

Network access policy for the container.

One of the following:

ContainerNetworkPolicyDisabled object { type }

type: "disabled"

Disable outbound network access. Always disabled.

ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }

allowed_domains: array of string

A list of allowed domains when type is allowlist.

type: "allowlist"

Allow outbound network access only to specified domains. Always allowlist.

domain_secrets: optional array of ContainerNetworkPolicyDomainSecret { domain, name, value }

Optional domain-scoped secrets for allowlisted domains.

domain: string

The domain associated with the secret.

minLength1

The name of the secret to inject for the domain.

minLength1

value: string

The secret value to inject for the domain.

minLength1

maxLength10485760

type: "code_interpreter"

The type of the code interpreter tool. Always code_interpreter.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

ProgrammaticToolCalling object { type }

type: "programmatic_tool_calling"

The type of the tool. Always programmatic_tool_calling.

ImageGeneration object { type, action, background, 9 more }

A tool that generates images using the GPT image models.

type: "image_generation"

The type of the image generation tool. Always image_generation.

action: optional "generate" or "edit" or "auto"

Whether to generate a new image or edit an existing image. Default: auto.

One of the following:

"generate"

"edit"

"auto"

background: optional "transparent" or "opaque" or "auto"

One of the following:

"transparent"

"opaque"

"auto"

input_fidelity: optional "high" or "low" or null

One of the following:

"high"

"low"

input_image_mask: optional object { file_id, image_url }

Optional mask for inpainting. Contains image_url (string, optional) and file_id (string, optional).

file_id: optional string

File ID for the mask image.

image_url: optional string

Base64-encoded mask image.

model: optional string or "gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more

One of the following:

string

"gpt-image-1" or "gpt-image-1-mini" or "gpt-image-1.5" or 6 more

One of the following:

"gpt-image-1"

"gpt-image-1-mini"

"gpt-image-1.5"

"gpt-image-2"

"gpt-image-2-2026-04-21"

"gpt-image-2.5-sunburst"

"gpt-image-2.5-sunburst-2026-09-08"

"gpt-image-2.5-flare"

"gpt-image-2.5-flare-2026-09-08"

moderation: optional "auto" or "low"

Moderation level for the generated image. Default: auto.

One of the following:

"auto"

"low"

output_compression: optional number

Compression level for the output image. Default: 100.

minimum0

maximum100

output_format: optional "png" or "webp" or "jpeg"

The output format of the generated image. One of png, webp, or jpeg. Default: png.

One of the following:

"png"

"webp"

"jpeg"

partial_images: optional number

Number of partial images to generate in streaming mode, from 0 (default value) to 3.

minimum0

maximum3

quality: optional "low" or "medium" or "high" or 3 more

One of the following:

"low"

"medium"

"high"

"xhigh"

"max"

"auto"

size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or "auto"

One of the following:

string

"1024x1024" or "1024x1536" or "1536x1024" or "auto"

One of the following:

"1024x1024"

"1024x1536"

"1536x1024"

"auto"

LocalShell object { type }

A tool that allows the model to execute shell commands in a local environment.

type: "local_shell"

The type of the local shell tool. Always local_shell.

Shell object { type, allowed_callers, environment }

A tool that allows the model to execute shell commands.

type: "shell"

The type of the shell tool. Always shell.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

environment: optional ContainerAuto { type, file_ids, memory_limit, 2 more } or LocalEnvironment { type, skills } or ContainerReference { container_id, type } or null

One of the following:

ContainerAuto object { type, file_ids, memory_limit, 2 more }

type: "container_auto"

Automatically creates a container for this request

file_ids: optional array of string

An optional list of uploaded files to make available to your code.

memory_limit: optional "1g" or "4g" or "16g" or "64g" or null

The memory limit for the container.

One of the following:

"1g"

"4g"

"16g"

"64g"

network_policy: optional ContainerNetworkPolicyDisabled { type } or ContainerNetworkPolicyAllowlist { allowed_domains, type, domain_secrets }

Network access policy for the container.

One of the following:

ContainerNetworkPolicyDisabled object { type }

type: "disabled"

Disable outbound network access. Always disabled.

ContainerNetworkPolicyAllowlist object { allowed_domains, type, domain_secrets }

allowed_domains: array of string

A list of allowed domains when type is allowlist.

type: "allowlist"

Allow outbound network access only to specified domains. Always allowlist.

domain_secrets: optional array of ContainerNetworkPolicyDomainSecret { domain, name, value }

Optional domain-scoped secrets for allowlisted domains.

domain: string

The domain associated with the secret.

minLength1

The name of the secret to inject for the domain.

minLength1

value: string

The secret value to inject for the domain.

minLength1

maxLength10485760

skills: optional array of SkillReference { skill_id, type, version } or InlineSkill { description, name, source, type }

An optional list of skills referenced by id or inline data.

One of the following:

SkillReference object { skill_id, type, version }

skill_id: string

The ID of the referenced skill.

minLength1

maxLength64

type: "skill_reference"

References a skill created with the /v1/skills endpoint.

version: optional string

Optional skill version. Use a positive integer or 'latest'. Omit for default.

InlineSkill object { description, name, source, type }

description: string

The description of the skill.

The name of the skill.

source: InlineSkillSource { data, media_type, type }

Inline skill payload

type: "inline"

Defines an inline skill for this request.

LocalEnvironment object { type, skills }

type: "local"

Use a local computer environment.

skills: optional array of LocalSkill { description, name, path }

An optional list of skills.

description: string

The description of the skill.

The name of the skill.

path: string

The path to the directory containing the skill.

ContainerReference object { container_id, type }

container_id: string

The ID of the referenced container.

type: "container_reference"

References a container created with the /v1/containers endpoint

Custom object { name, type, allowed_callers, 4 more }

A custom tool that processes input using a specified format. Learn more about custom tools

The name of the custom tool, used to identify it in tool calls.

type: "custom"

The type of the custom tool. Always custom.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

async: optional boolean

Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

defer_loading: optional boolean

Whether this tool should be deferred and discovered via tool search.

description: optional string

Optional description of the custom tool, used to provide more context.

format: optional CustomToolInputFormat

The input format for the custom tool. Default is unconstrained text.

Namespace object { description, name, tools, type }

Groups function/custom tools under a shared namespace.

description: string

A description of the namespace shown to the model.

The namespace name used in tool calls (for example, crm).

minLength1

tools: array of Function { name, type, allowed_callers, 6 more } or Custom { name, type, allowed_callers, 4 more }

The function/custom tools available inside this namespace.

One of the following:

Function object { name, type, allowed_callers, 6 more }

minLength1

maxLength128

type: "function"

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

async: optional boolean

Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

defer_loading: optional boolean

Whether this function should be deferred and discovered via tool search.

description: optional string or null

output_schema: optional map[unknown] or null

A JSON Schema describing the JSON value encoded in string outputs for this function tool. This does not describe content-array outputs.

parameters: optional unknown or null

strict: optional boolean or null

Whether to enforce strict parameter validation. If omitted, Responses attempts to use strict validation when the schema is compatible, and falls back to non-strict validation otherwise.

Custom object { name, type, allowed_callers, 4 more }

A custom tool that processes input using a specified format. Learn more about custom tools

The name of the custom tool, used to identify it in tool calls.

type: "custom"

The type of the custom tool. Always custom.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

async: optional boolean

Whether the tool response can be returned asynchronously versus immediately returned on next response creation.

defer_loading: optional boolean

Whether this tool should be deferred and discovered via tool search.

description: optional string

Optional description of the custom tool, used to provide more context.

format: optional CustomToolInputFormat

The input format for the custom tool. Default is unconstrained text.

type: "namespace"

The type of the tool. Always namespace.

ToolSearch object { type, description, execution, parameters }

Hosted or BYOT tool search configuration for deferred tools.

type: "tool_search"

The type of the tool. Always tool_search.

description: optional string or null

Description shown to the model for a client-executed tool search tool.

execution: optional "server" or "client"

Whether tool search is executed by the server or by the client.

One of the following:

"server"

"client"

parameters: optional unknown or null

Parameter schema for a client-executed tool search tool.

WebSearchPreview object { type, search_content_types, search_context_size, user_location }

This tool searches the web for relevant results to use in a response. Learn more about the web search tool.

type: "web_search_preview" or "web_search_preview_2025_03_11"

The type of the web search tool. One of web_search_preview or web_search_preview_2025_03_11.

One of the following:

"web_search_preview"

"web_search_preview_2025_03_11"

search_content_types: optional array of "text" or "image"

One of the following:

"text"

"image"

search_context_size: optional "low" or "medium" or "high"

High level guidance for the amount of context window space to use for the search. One of low, medium, or high. medium is the default.

One of the following:

"low"

"medium"

"high"

user_location: optional object { type, city, country, 2 more } or null

type: "approximate"

The type of location approximation. Always approximate.

city: optional string or null

Free text input for the city of the user, e.g. San Francisco.

country: optional string or null

The two-letter ISO country code of the user, e.g. US.

region: optional string or null

Free text input for the region of the user, e.g. California.

timezone: optional string or null

The IANA timezone of the user, e.g. America/Los_Angeles.

ApplyPatch object { type, allowed_callers }

Allows the assistant to create, delete, or update files using unified diffs.

type: "apply_patch"

The type of the tool. Always apply_patch.

allowed_callers: optional array of "direct" or "programmatic" or null

The tool invocation context(s).

One of the following:

"direct"

"programmatic"

type: "additional_tools"

The item type. Always additional_tools.

id: optional string or null

The unique ID of this additional tools item.

ConfigurationUpdate object { type, id, reasoning }

An update to the conversation's response configuration. The configuration remains in effect for subsequent responses until it is replaced by another configuration update.

type: "configuration_update"

The item type. Always configuration_update.

id: optional string or null

The unique ID of the configuration update item.

reasoning: optional object { effort }

Updates to reasoning configuration. Only effort is supported.

effort: optional ReasoningEffort or null

The reasoning effort to use for subsequent responses until another configuration update replaces it.

Reasoning object { id, summary, type, 3 more }

A description of the chain of thought used by a reasoning model while generating a response. Be sure to include these items in your input to the Responses API for subsequent turns of a conversation if you are manually managing context.

id: string

The unique identifier of the reasoning content.

summary: array of SummaryTextContent { text, type }

Reasoning summary content.

text: string

A summary of the reasoning output from the model so far.

type: "summary_text"

The type of the object. Always summary_text.

type: "reasoning"

The type of the object. Always reasoning.

content: optional array of object { text, type }

Reasoning text content.

text: string

The reasoning text from the model.

type: "reasoning_text"

The type of the reasoning text. Always reasoning_text.

encrypted_content: optional string or null

The encrypted content of the reasoning item. This is populated by default for reasoning items returned by POST /v1/responses and WebSocket response.create requests.

When streaming, use the completed reasoning item and its encrypted_content from the response.output_item.done event in subsequent requests. The encrypted_content in response.output_item.added may be incomplete. This is especially important when store is false or when using Zero Data Retention.

status: optional "in_progress" or "completed" or "incomplete"

The status of the item. One of in_progress, completed, or incomplete. Populated when items are returned via API.

One of the following:

"in_progress"

"completed"

"incomplete"

Compaction object { encrypted_content, type, id }

A compaction item generated by the v1/responses/compact API.

encrypted_content: string

The encrypted content of the compaction summary.

maxLength104857600

type: "compaction"

The type of the item. Always compaction.

id: optional string or null

The ID of the compaction item.

ImageGenerationCall object { id, result, status, 8 more }

An image generation request made by the model.

id: string

The unique ID of the image generation call.

result: string or null

The generated image encoded in base64.

status: "in_progress" or "completed" or "generating" or "failed"

The status of the image generation call.

One of the following:

"in_progress"

"completed"

"generating"

"failed"

type: "image_generation_call"

The type of the image generation call. Always image_generation_call.

action: optional "generate" or "edit" or "auto" or null

The action used for image generation.

One of the following:

"generate"

"edit"

"auto"

background: optional "transparent" or "opaque" or "auto" or null

The background setting used for generation.

One of the following:

"transparent"

"opaque"

"auto"

error: optional object { code, message, param, 2 more }

Details of a moderation failure, when available.

code: string or null

The error code.

message: string

A description of the error.

param: string or null

The related request parameter.

type: string

The error type.

moderation_details: optional object { categories, moderation_stage }

Public details for a moderation_blocked error, when available.

categories: array of string

Public moderation categories. New category values may be added over time.

moderation_stage: "input" or "output" or "unknown"

The stage where moderation blocked the image generation call.

One of the following:

"input"

"output"

"unknown"

output_format: optional "png" or "webp" or "jpeg" or null

The output format used for generation.

One of the following:

"png"

"webp"

"jpeg"

quality: optional "low" or "medium" or "high" or 3 more or null

The quality of the image generated by the image generation tool call. One of low, medium, high, xhigh, max, or auto.

One of the following:

"low"

"medium"

"high"

"xhigh"

"max"

"auto"

revised_prompt: optional string or null

The prompt that was used after any model prompt rewriting.

size: optional string or "1024x1024" or "1024x1536" or "1536x1024" or null

The image dimensions as a WIDTHxHEIGHT string, for example 1536x864.

One of the following:

string

"1024x1024" or "1024x1536" or "1536x1024"

The image dimensions as a WIDTHxHEIGHT string, for example 1536x864.

One of the following:

"1024x1024"

"1024x1536"

"1536x1024"

CodeInterpreterCall object { id, code, container_id, 3 more }

A tool call to run code.

id: string

The unique ID of the code interpreter tool call.

code: string or null

The code to run, or null if not available.

container_id: string

The ID of the container used to run the code.

outputs: array of Logs { logs, type } or Image { type, url } or null

The outputs generated by the code interpreter, such as logs or images. Can be null if no outputs are available.

One of the following:

Logs object { logs, type }

The logs output from the code interpreter.

logs: string

The logs output from the code interpreter.

type: "logs"

The type of the output. Always logs.

Image object { type, url }

The image output from the code interpreter.

type: "image"

The type of the output. Always image.

url: string

The URL of the image output from the code interpreter.

formaturi

status: "in_progress" or "completed" or "incomplete" or 2 more

The status of the code interpreter tool call. Valid values are in_progress, completed, incomplete, interpreting, and failed.

One of the following:

"in_progress"

"completed"

"incomplete"

"interpreting"

"failed"

type: "code_interpreter_call"

The type of the code interpreter tool call. Always code_interpreter_call.

LocalShellCall object { id, action, call_id, 2 more }

A tool call to run a command on the local shell.

id: string

The unique ID of the local shell call.

action: object { command, env, type, 3 more }

Execute a shell command on the server.

command: array of string

The command to run.

env: map[string]

Environment variables to set for the command.

type: "exec"

The type of the local shell action. Always exec.

timeout_ms: optional number or null

Optional timeout in milliseconds for the command.

user: optional string or null

Optional user to run the command as.

working_directory: optional string or null

Optional working directory to run the command in.

call_id: string

The unique ID of the local shell tool call generated by the model.

status: "in_progress" or "completed" or "incomplete"

The status of the local shell call.

One of the following:

"in_progress"

"completed"

"incomplete"

type: "local_shell_call"

The type of the local shell call. Always local_shell_call.

LocalShellCallOutput object { id, output, type, status }

The output of a local shell tool call.

id: string

The unique ID of the local shell tool call generated by the model.

output: string

A JSON string of the output of the local shell tool call.

type: "local_shell_call_output"

The type of the local shell tool call output. Always local_shell_call_output.

status: optional "in_progress" or "completed" or "incomplete" or null

The status of the item. One of in_progress, completed, or incomplete.

One of the following:

"in_progress"

"completed"

"incomplete"

ShellCall object { action, call_id, type, 4 more }

A tool representing a request to execute one or more shell commands.

action: object { commands, max_output_length, timeout_ms }

The shell commands and limits that describe how to run the tool call.

commands: array of string

Ordered shell commands for the execution environment to run.

max_output_length: optional number or null

Maximum number of UTF-8 characters to capture from combined stdout and stderr output.

timeout_ms: optional number or null

Maximum wall-clock time in milliseconds to allow the shell commands to run.

call_id: string

The unique ID of the shell tool call generated by the model.

minLength1

maxLength64

type: "shell_call"

The type of the item. Always shell_call.

id: optional string or null

The unique ID of the shell tool call. Populated when this item is returned via API.

caller: optional Direct { type } or Program { caller_id, type } or null

The execution context that produced this tool call.

One of the following:

Direct object { type }

type: "direct"

The caller type. Always direct.

Program object { caller_id, type }

caller_id: string

The call ID of the program item that produced this tool call.

minLength1

maxLength64

type: "program"

The caller type. Always program.

environment: optional LocalEnvironment { type, skills } or ContainerReference { container_id, type } or null

The environment to execute the shell commands in.

One of the following:

LocalEnvironment object { type, skills }

type: "local"

Use a local computer environment.

skills: optional array of LocalSkill { description, name, path }

An optional list of skills.

description: string

The description of the skill.

The name of the skill.

path: string

The path to the directory containing the skill.

ContainerReference object { container_id, type }

container_id: string

The ID of the referenced container.

type: "container_reference"

References a container created with the /v1/containers endpoint

status: optional "in_progress" or "completed" or "incomplete" or null

The status of the shell call. One of in_progress, completed, or incomplete.

One of the following:

"in_progress"

"completed"

"incomplete"

ShellCallOutput object { call_id, output, type, 4 more }

The streamed output items emitted by a shell tool call.

call_id: string

The unique ID of the shell tool call generated by the model.

minLength1

maxLength64

output: array of ResponseFunctionShellCallOutputContent { outcome, stderr, stdout }

Captured chunks of stdout and stderr output, along with their associated outcomes.

outcome: Timeout { type } or Exit { exit_code, type }

The exit or timeout outcome associated with this shell call.

One of the following:

Timeout object { type }

Indicates that the shell call exceeded its configured time limit.

type: "timeout"

The outcome type. Always timeout.

Exit object { exit_code, type }

Indicates that the shell commands finished and returned an exit code.

exit_code: number

The exit code returned by the shell process.

type: "exit"

The outcome type. Always exit.

stderr: string

Captured stderr output for the shell call.

maxLength10485760

stdout: string

Captured stdout output for the shell call.

maxLength10485760

type: "shell_call_output"

The type of the item. Always shell_call_output.

id: optional string or null

The unique ID of the shell tool call output. Populated when this item is returned via API.

caller: optional Direct { type } or Program { caller_id, type } or null

The execution context that produced this tool call.

One of the following:

Direct object { type }

type: "direct"

The caller type. Always direct.

Program object { caller_id, type }

caller_id: string

The call ID of the program item that produced this tool call.

minLength1

maxLength64

type: "program"

The caller type. Always program.

max_output_length: optional number or null

The maximum number of UTF-8 characters captured for this shell call's combined output.

status: optional "in_progress" or "completed" or "incomplete" or null

The status of the shell call output.

One of the following:

"in_progress"

"completed"

"incomplete"

ApplyPatchCall object { call_id, operation, status, 3 more }

A tool call representing a request to create, delete, or update files using diff patches.

call_id: string

The unique ID of the apply patch tool call generated by the model.

minLength1

maxLength64

operation: CreateFile { diff, path, type } or DeleteFile { path, type } or UpdateFile { diff, path, type }

The specific create, delete, or update instruction for the apply_patch tool call.

One of the following:

CreateFile object { diff, path, type }

Instruction for creating a new file via the apply_patch tool.

diff: string

Unified diff content to apply when creating the file.

maxLength10485760

path: string

Path of the file to create relative to the workspace root.

minLength1

type: "create_file"

The operation type. Always create_file.

DeleteFile object { path, type }

Instruction for deleting an existing file via the apply_patch tool.

path: string

Path of the file to delete relative to the workspace root.

minLength1

type: "delete_file"

The operation type. Always delete_file.

UpdateFile object { diff, path, type }

Instruction for updating an existing file via the apply_patch tool.

diff: string

Unified diff content to apply to the existing file.

maxLength10485760

path: string

Path of the file to update relative to the workspace root.

minLength1

type: "update_file"

The operation type. Always update_file.

status: "in_progress" or "completed"

The status of the apply patch tool call. One of in_progress or completed.

One of the following:

"in_progress"

"completed"

type: "apply_patch_call"

The type of the item. Always apply_patch_call.

id: optional string or null

The unique ID of the apply patch tool call. Populated when this item is returned via API.

caller: optional Direct { type } or Program { caller_id, type } or null

The execution context that produced this tool call.

One of the following:

Direct object { type }

type: "direct"

The caller type. Always direct.

Program object { caller_id, type }

caller_id: string

The call ID of the program item that produced this tool call.

minLength1

maxLength64

type: "program"

The caller type. Always program.

ApplyPatchCallOutput object { call_id, status, type, 3 more }

The streamed output emitted by an apply patch tool call.

call_id: string

The unique ID of the apply patch tool call generated by the model.

minLength1

maxLength64

status: "completed" or "failed"

The status of the apply patch tool call output. One of completed or failed.

One of the following:

"completed"

"failed"

type: "apply_patch_call_output"

The type of the item. Always apply_patch_call_output.

id: optional string or null

The unique ID of the apply patch tool call output. Populated when this item is returned via API.

caller: optional Direct { type } or Program { caller_id, type } or null

The execution context that produced this tool call.

One of the following:

Direct object { type }

type: "direct"

The caller type. Always direct.

Program object { caller_id, type }

caller_id: string

The call ID of the program item that produced this tool call.

minLength1

maxLength64

type: "program"

The caller type. Always program.

output: optional string or null

Optional human-readable log text from the apply patch tool (e.g., patch results or errors).

maxLength10485760

McpListTools object { id, server_label, tools, 2 more }

A list of tools available on an MCP server.

id: string

The unique ID of the list.

server_label: string

The label of the MCP server.

tools: array of object { input_schema, name, annotations, description }

The tools available on the server.

input_schema: unknown

The JSON schema describing the tool's input.

The name of the tool.

annotations: optional unknown or null

Additional annotations about the tool.

description: optional string or null

The description of the tool.

type: "mcp_list_tools"

The type of the item. Always mcp_list_tools.

error: optional string or null

Error message if the server could not list tools.

McpApprovalRequest object { id, arguments, name, 2 more }

A request for human approval of a tool invocation.

id: string

The unique ID of the approval request.

arguments: string

A JSON string of arguments for the tool.

The name of the tool to run.

server_label: string

The label of the MCP server making the request.

type: "mcp_approval_request"

The type of the item. Always mcp_approval_request.

McpApprovalResponse object { approval_request_id, approve, type, 2 more }

A response to an MCP approval request.

approval_request_id: string

The ID of the approval request being answered.

approve: boolean

Whether the request was approved.

type: "mcp_approval_response"

The type of the item. Always mcp_approval_response.

id: optional string or null

The unique ID of the approval response

reason: optional string or null

Optional reason for the decision.

McpCall object { id, arguments, name, 6 more }

An invocation of a tool on an MCP server.

id: string

The unique ID of the tool call.

arguments: string

A JSON string of the arguments passed to the tool.

The name of the tool that was run.

server_label: string

The label of the MCP server running the tool.

type: "mcp_call"

The type of the item. Always mcp_call.

approval_request_id: optional string or null

Unique identifier for the MCP tool call approval request. Include this value in a subsequent mcp_approval_response input to approve or reject the corresponding tool call.

error: optional McpToolCallError or null

The error from the tool call, if any.

output: optional string or null

The output from the tool call.

status: optional "in_progress" or "completed" or "incomplete" or 2 more

The status of the tool call. One of in_progress, completed, incomplete, calling, or failed.

One of the following:

"in_progress"

"completed"

"incomplete"

"calling"

"failed"

CustomToolCallOutput object { call_id, output, type, 2 more }

The output of a custom tool call from your code, being sent back to the model.

call_id: string

The call ID, used to map this custom tool call output to a custom tool call.

output: string or array of ResponseInputText { text, type, prompt_cache_breakpoint } or ResponseInputImage { detail, type, file_id, 2 more } or ResponseInputFile { type, detail, file_data, 4 more }

The output from the custom tool call generated by your code. Can be a string or an list of output content.

One of the following:

StringOutput = string

A string of the output of the custom tool call.

OutputContentList = array of ResponseInputText { text, type, prompt_cache_breakpoint } or ResponseInputImage { detail, type, file_id, 2 more } or ResponseInputFile { type, detail, file_data, 4 more }

Text, image, or file output of the custom tool call.

One of the following:

ResponseInputText object { text, type, prompt_cache_breakpoint }

A text input to the model.

text: string

The text input to the model.

type: "input_text"

The type of the input item. Always input_text.

prompt_cache_breakpoint: optional object { mode }

Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

mode: "explicit"

The breakpoint mode. Always explicit.

ResponseInputImage object { detail, type, file_id, 2 more }

An image input to the model. Learn about image inputs.

detail: ImageDetail

The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.

type: "input_image"

The type of the input item. Always input_image.

file_id: optional string or null

The ID of the file to be sent to the model.

image_url: optional string or null

The URL of the image to be sent to the model. A fully qualified URL or base64 encoded image in a data URL.

formaturi

prompt_cache_breakpoint: optional object { mode }

Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

mode: "explicit"

The breakpoint mode. Always explicit.

ResponseInputFile object { type, detail, file_data, 4 more }

A file input to the model.

type: "input_file"

The type of the input item. Always input_file.

detail: optional "auto" or "low" or "high"

One of the following:

"auto"

"low"

"high"

file_data: optional string

The content of the file to be sent to the model.

file_id: optional string or null

The ID of the file to be sent to the model.

file_url: optional string

The URL of the file to be sent to the model.

formaturi

filename: optional string

The name of the file to be sent to the model.

prompt_cache_breakpoint: optional object { mode }

Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's prompt_cache_options.ttl; the boundary is not rounded to a token block.

mode: "explicit"

The breakpoint mode. Always explicit.

type: "custom_tool_call_output"

The type of the custom tool call output. Always custom_tool_call_output.

id: optional string

The unique ID of the custom tool call output in the OpenAI platform.

caller: optional Direct { type } or Program { caller_id, type } or null

The execution context that produced this tool call.

One of the following:

Direct object { type }

type: "direct"

The caller type. Always direct.

Program object { caller_id, type }

caller_id: string

The call ID of the program item that produced this tool call.

minLength1

maxLength64

type: "program"

The caller type. Always program.

CustomToolCall object { call_id, input, name, 5 more }

A call to a custom tool created by the model.

call_id: string

An identifier used to map this custom tool call to a tool call output.

input: string

The input for the custom tool call generated by the model.

The name of the custom tool being called.

type: "custom_tool_call"

The type of the custom tool call. Always custom_tool_call.

id: optional string

The unique ID of the custom tool call in the OpenAI platform.

async: optional boolean

Whether the custom tool call runs asynchronously.

caller: optional Direct { type } or Program { caller_id, type } or null

The execution context that produced this tool call.

One of the following:

Direct object { type }

type: "direct"

Program object { caller_id, type }

caller_id: string

The call ID of the program item that produced this tool call.

type: "program"

namespace: optional string

The namespace of the custom tool being called.

CompactionTrigger object { type, id }

Compacts the current context. Must be the final input item.

type: "compaction_trigger"

The type of the item. Always compaction_trigger.

id: optional string or null

The unique ID of this compaction trigger.

ItemReference object { id, type }

An internal identifier for an item to reference.

id: string

The ID of the item to reference.

type: optional "item_reference" or null

The type of item to reference. Always item_reference.

Program object { id, call_id, code, 2 more }

id: string

The unique ID of this program item.

call_id: string

The stable call ID of the program item.

minLength1

maxLength64

code: string

The JavaScript source executed by programmatic tool calling.

maxLength10485760

fingerprint: string

Opaque program replay fingerprint that must be round-tripped.

maxLength10485760

type: "program"

The item type. Always program.

ProgramOutput object { id, call_id, result, 2 more }

id: string

The unique ID of this program output item.

call_id: string

The call ID of the program item.

minLength1

maxLength64

result: string

The result produced by the program item.

maxLength10485760

status: "completed" or "incomplete"

The terminal status of the program output.

One of the following:

"completed"

"incomplete"

type: "program_output"

The item type. Always program_output.

type: "response.item.create"

The Live client event type. Always response.item.create.

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

OBJECT

response.item.create

{
  "type": "response.item.create",
  "event_id": "evt_item_001",
  "item": {
    "type": "message",
    "role": "user",
    "content": [
      {
        "type": "input_text",
        "text": "Please check for a table for two at 7 PM."
      }
    ]
  }
}

response.create

Request a response from the Live session’s Responses backend, or continue a delegated response waiting for tool results. Requires Responses delegation.

type: "response.create"

The Live client event type. Always response.create.

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

session.close

Request that the Live session close. The terminal session.closed event contains the close reason and final usage.

type: "session.close"

The Live client event type. Always session.close.

event_id: optional string or null

Optional client identifier for correlating this command with a server event's client_event_id or error.client_event_id.

maxLength512

Server events

transport.ringing

The outbound SIP provider leg is ringing or providing early media. Delivered only to sideband observers.

event_id: string

session_id: string

The canonical Live session ID.

type: "transport.ringing"

transport.answered

The outbound SIP provider leg answered and media is established. Delivered only to sideband observers.

event_id: string

session_id: string

The canonical Live session ID.

type: "transport.answered"

transport.failed

An asynchronous outbound SIP setup failure. Delivered only to sideband observers.

error: object { code, message, type, param }

code: string

The call setup failure code.

message: string

type: "call_error"

param: optional string

The parameter related to the error, if any. Empty when no parameter applies.

event_id: string

session_id: string

The canonical Live session ID.

type: "transport.failed"

OBJECT

transport.failed

{
  "type": "transport.failed",
  "event_id": "event_call_4",
  "session_id": "live_u0_123",
  "error": {
    "type": "call_error",
    "code": "provider_invite_failed",
    "message": "provider rejected the call",
    "param": ""
  }
}

session.started

Returned when a Live session has started. Contains the resolved session configuration, including server defaults.

event_id: string

The unique ID of the Live server event.

session: SessionResource { id, expires_at, model, 8 more }

The resolved Live session configuration and server-assigned session metadata.

type: "session.started"

The event type, always session.started.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

OBJECT

session.started

{
  "type": "session.started",
  "event_id": "evt_started_001",
  "client_event_id": "evt_start_001",
  "session": {
    "id": "live_abc123",
    "model": "gpt-live-1",
    "status": "active",
    "expires_at": 1788555600,
    "instructions": "Help the caller plan a restaurant reservation. Confirm details before booking.",
    "input": [],
    "audio": {
      "format": {
        "type": "audio/pcm",
        "rate": 24000
      },
      "output": {
        "voice": "marin"
      }
    },
    "delegation": {
      "type": "client"
    }
  }
}

session.updated

Returned when a Live session update is accepted. Contains the resolved session configuration after the update.

event_id: string

The unique ID of the Live server event.

session: SessionResource { id, expires_at, model, 8 more }

The resolved Live session configuration and server-assigned session metadata.

type: "session.updated"

The event type, always session.updated.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

OBJECT

session.updated

{
  "type": "session.updated",
  "event_id": "evt_updated_001",
  "client_event_id": "evt_update_001",
  "session": {
    "id": "live_def456",
    "model": "gpt-live-1",
    "status": "active",
    "expires_at": 1788555600,
    "instructions": "Help the caller plan a restaurant reservation. Confirm details before booking.",
    "input": [],
    "audio": {
      "format": {
        "type": "audio/pcm",
        "rate": 24000
      },
      "output": {
        "voice": "marin"
      }
    },
    "delegation": {
      "type": "responses",
      "responses": {
        "model": "gpt-6-astra",
        "instructions": "Check restaurant availability. Ask before confirming a booking.",
        "max_output_tokens": 1024,
        "tools": []
      }
    }
  }
}

session.input_audio.muted

Acknowledges a session.input_audio.mute command. An independent input mute sends silence to the model and preserves sideband audio reflection. Fixture playback does not emit this event.

event_id: string

The unique ID of the Live server event.

type: "session.input_audio.muted"

The event type, always session.input_audio.muted.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

session.input_audio.unmuted

Acknowledges a session.input_audio.unmute command, removing the independent input mute. An active audio fixture still pauses inference until playback finishes. Fixture playback does not emit this event.

event_id: string

The unique ID of the Live server event.

type: "session.input_audio.unmuted"

The event type, always session.input_audio.unmuted.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

session.output_audio.fixture.playing

The existing playback path has accepted the validated audio fixture. This acknowledgment does not identify when the listener receives the first sample. The optional client_event_id echoes the play command.

event_id: string

The unique ID of the Live server event.

type: "session.output_audio.fixture.playing"

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

session.output_audio.fixture.played

The complete audio fixture has drained from the server mixer and normal inference has resumed. Transport buffers may still contain audio; this event does not prove listener-side completion. The optional client_event_id echoes the play command. This event is not emitted for canceled or failed playback.

event_id: string

The unique ID of the Live server event.

type: "session.output_audio.fixture.played"

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

session.instructions.appended

Returned when a session.instructions.append command is accepted into the Live session timeline. Acknowledges the appended instructions without guaranteeing that the model has acted on them.

end_ms: number

The end of this event on the Live session timeline, in milliseconds from the beginning of the session. For appended context, this can equal start_ms.

event_id: string

The unique ID of the Live server event.

start_ms: number

The start of this event on the Live session timeline, in milliseconds from the beginning of the session.

type: "session.instructions.appended"

The event type, always session.instructions.appended.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

OBJECT

session.instructions.appended

{
  "type": "session.instructions.appended",
  "event_id": "evt_instructions_002",
  "client_event_id": "evt_instructions_001",
  "start_ms": 1200,
  "end_ms": 1400
}

session.thinking.appended

Returned when a session.thinking.append command is accepted into the Live session timeline. Acknowledges the added reasoning context without guaranteeing any spoken output.

end_ms: number

The end of this event on the Live session timeline, in milliseconds from the beginning of the session. For appended context, this can equal start_ms.

event_id: string

The unique ID of the Live server event.

start_ms: number

The start of this event on the Live session timeline, in milliseconds from the beginning of the session.

type: "session.thinking.appended"

The event type, always session.thinking.appended.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

session.commentary.appended

Returned when a session.commentary.append command is accepted into the Live session timeline. Acknowledges the added commentary without guaranteeing exact wording or completed audio playback.

end_ms: number

The end of this event on the Live session timeline, in milliseconds from the beginning of the session. For appended context, this can equal start_ms.

event_id: string

The unique ID of the Live server event.

start_ms: number

The start of this event on the Live session timeline, in milliseconds from the beginning of the session.

type: "session.commentary.appended"

The event type, always session.commentary.appended.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

session.output_audio.delta

An audio chunk generated by the Live model. Decode and play primary WebSocket chunks in delivery order using the configured session audio format. Sideband connections receive reflected output audio with timestamps.

delta: string

Base64-encoded raw audio. Primary WebSocket events use the session's configured format; reflected sideband events use mono PCM16LE at 24 kHz.

type: "session.output_audio.delta"

The event type, always session.output_audio.delta.

end_ms: optional number

Exclusive session-relative end in milliseconds. Required on reflected sideband events; omitted on the primary WebSocket. Dropped output frames leave gaps between reflected ranges.

start_ms: optional number

Inclusive session-relative start in milliseconds. Required on reflected sideband events; omitted on the primary WebSocket.

session.input_transcript.delta

A transcript fragment for user input audio in the Live session. Accumulate fragments in delivery order; these events do not define complete turns or include a transcript-done event.

delta: string

The transcript text fragment for the audio in this time range. Append fragments in delivery order to build the transcript.

end_ms: number

The end of this event on the Live session timeline, in milliseconds from the beginning of the session. For appended context, this can equal start_ms.

event_id: string

The unique ID of the Live server event.

start_ms: number

The start of this event on the Live session timeline, in milliseconds from the beginning of the session.

type: "session.input_transcript.delta"

The event type, always session.input_transcript.delta.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

OBJECT

session.input_transcript.delta

{
  "type": "session.input_transcript.delta",
  "event_id": "evt_input_transcript_001",
  "delta": "A table for two at seven, please.",
  "start_ms": 1600,
  "end_ms": 3400
}

session.output_transcript.delta

A transcript fragment for assistant output audio in the Live session. Accumulate fragments in delivery order; these events do not define complete turns or include a transcript-done event.

delta: string

The transcript text fragment for the audio in this time range. Append fragments in delivery order to build the transcript.

end_ms: number

The end of this event on the Live session timeline, in milliseconds from the beginning of the session. For appended context, this can equal start_ms.

event_id: string

The unique ID of the Live server event.

start_ms: number

The start of this event on the Live session timeline, in milliseconds from the beginning of the session.

type: "session.output_transcript.delta"

The event type, always session.output_transcript.delta.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

OBJECT

session.output_transcript.delta

{
  "type": "session.output_transcript.delta",
  "event_id": "evt_output_transcript_001",
  "delta": "Would you like me to reserve that table?",
  "start_ms": 5400,
  "end_ms": 7200
}

session.delegation.created

Returned when the Live model delegates work to your application, Responses, or Decisions. Contains delegation metadata and the position on the session timeline where the work was delegated.

delegation: object { id, target, type, response_id }

The delegated work identifier and destination. This object contains metadata, not the task text.

id: string

The unique ID of the delegation. Use this as delegation_id when replying to client-owned work or correlating backend events.

target: "client" or "responses" or "decisions"

Where the Live model delegated the work: client, responses, or decisions.

One of the following:

"client" or "responses" or "decisions"

Where the Live model delegated the work: client, responses, or decisions.

One of the following:

"client"

"responses"

"decisions"

type: "delegation"

The object type, always delegation.

response_id: optional string

The ID of the Responses API response associated with a Responses delegation. Omitted for client delegations.

event_id: string

The unique ID of the Live server event.

offset_ms: number

The position on the Live session timeline where the delegation was created, in milliseconds from the beginning of the session.

type: "session.delegation.created"

The event type, always session.delegation.created.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

OBJECT

session.delegation.created

{
  "type": "session.delegation.created",
  "event_id": "evt_delegation_001",
  "offset_ms": 3600,
  "delegation": {
    "id": "del_abc123",
    "type": "delegation",
    "target": "client"
  }
}

decision.event

A Decision API event for a Live delegation or transcript trigger. For a trigger, source_event_id identifies the transcript delta that caused this evaluation.

delegation_id: string or null

The Live delegation ID, or null for a transcript trigger.

event: object { decision, type }

decision: Decision { answers, model, usage }

The native Decision API response, including its model, ordered answers, and token usage.

type: "decision.completed"

event_id: string

The unique ID of this Live server event.

source_event_id: string or null

The event_id of the transcript delta that fired this trigger, or null for a delegation.

source_event_type: "session.input_transcript.delta" or "session.output_transcript.delta" or null

One of the following:

"session.input_transcript.delta" or "session.output_transcript.delta"

One of the following:

"session.input_transcript.delta"

"session.output_transcript.delta"

trigger_id: string or null

The Live-assigned ID of this trigger firing, or null for a delegation.

type: "decision.event"

OBJECT

decision.event

{
  "type": "decision.event",
  "event_id": "evt_decision_1",
  "delegation_id": "del_abc123",
  "trigger_id": null,
  "source_event_type": null,
  "source_event_id": null,
  "event": {
    "type": "decision.completed",
    "decision": {
      "model": "gpt-6-luna",
      "answers": [
        {
          "type": "predicate",
          "name": "damaged",
          "probability": 0.95
        }
      ],
      "usage": {
        "input_tokens": 42,
        "input_tokens_details": {
          "cached_tokens": 0,
          "cache_write_tokens": 0
        },
        "output_tokens": 0,
        "output_tokens_details": {
          "reasoning_tokens": 0
        },
        "total_tokens": 42
      }
    }
  }
}

decision.item.created

An input item was appended for future decisions.

event_id: string

The unique ID of the Live server event.

item: object { id, content, role, type }

id: string

The ID of the stored Decision input item.

content: string or array of InputText { text, type } or InputImage { image_url, type, detail }

The text input to the model.

One of the following:

string

The text input to the model.

array of InputText { text, type } or InputImage { image_url, type, detail }

One of the following:

InputText object { text, type }

text: string

The text input to the model.

maxLength10485760

type: "input_text"

The type of the input item. Always input_text.

InputImage object { image_url, type, detail }

image_url: string

An inline image encoded as a data URL.

maxLength20971520

type: "input_image"

The type of the input item. Always input_image.

detail: optional "low" or "high" or "auto" or "original" or null

The detail level of the image to be sent to the model. One of high, low, auto, or original. Defaults to auto.

One of the following:

"low"

"high"

"auto"

"original"

role: "user"

The message role. Always user.

type: optional "message"

type: "decision.item.created"

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

OBJECT

decision.item.created

{
  "type": "decision.item.created",
  "event_id": "evt_item_1",
  "client_event_id": "browser_1",
  "item": {
    "id": "msg_browser_1",
    "type": "message",
    "role": "user",
    "content": [
      {
        "type": "input_text",
        "text": "Wikipedia is open."
      }
    ]
  }
}

decision.item.deleted

An input item was removed from future decisions.

event_id: string

The unique ID of the Live server event.

item_id: string

type: "decision.item.deleted"

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

response.event

A streaming Responses API event from a backend delegated to by the Live session. Use the outer delegation_id to associate the nested stream with its Live delegation.

event: map[unknown]

The nested Responses streaming event. Dispatch on its type field. Response lifecycle snapshots omit input and clear instructions, tools, and output to keep messages small; consume granular output events for the generated content.

event_id: string

The unique ID of the Live server event.

type: "response.event"

The event type, always response.event.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

delegation_id: optional string or null

The Live delegation associated with the nested Responses event. May be null or omitted when the event cannot be correlated with a delegation.

OBJECT

response.event

{
  "type": "response.event",
  "event_id": "evt_response_002",
  "delegation_id": "del_responses123",
  "event": {
    "type": "response.output_text.delta",
    "item_id": "msg_abc123",
    "output_index": 0,
    "content_index": 0,
    "delta": "An outdoor table is available at 7 PM.",
    "sequence_number": 3,
    "logprobs": []
  }
}

session.usage.updated

Reports cumulative Live audio usage and, when available, the most recent context-window usage. Delegated Responses token usage is reported separately in response.event events.

event_id: string

The unique ID of the Live server event.

type: "session.usage.updated"

The event type, always session.usage.updated.

usage: SessionUsage { seconds }

The cumulative Live audio usage so far.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

context_window: optional object { usage_ratio }

The latest measured Live context-window usage. Omitted when the context limit is unknown.

usage_ratio: number

The latest active context token count divided by the Live model context limit. Can decrease after compaction and may lag between measured audio frames.

session.closed

Returned after the Live session finishes finalizing, with the close reason, final session snapshot, and cumulative audio usage. A connection closing without this event does not confirm successful finalization.

event_id: string

The unique ID of the Live server event.

reason: "close_requested" or "expired" or "content" or 2 more

Why the Live session ended: close_requested for an application close or hangup request, expired for the session duration limit, content for a safety filter, remote_hangup for a graceful remote disconnect, or connection_lost for an unexpected primary or upstream disconnection.

One of the following:

"close_requested" or "expired" or "content" or 2 more

One of the following:

"close_requested"

"expired"

"content"

"remote_hangup"

"connection_lost"

session: SessionResource { id, expires_at, model, 8 more }

The resolved Live session configuration and server-assigned session metadata.

type: "session.closed"

The event type, always session.closed.

usage: SessionUsage { seconds }

The final cumulative Live audio usage after session finalization.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

OBJECT

session.closed

{
  "type": "session.closed",
  "event_id": "evt_closed_001",
  "client_event_id": "evt_close_001",
  "reason": "close_requested",
  "session": {
    "id": "live_abc123",
    "model": "gpt-live-1",
    "status": "active",
    "expires_at": 1788555600,
    "instructions": "Help the caller plan a restaurant reservation. Confirm details before booking.",
    "input": [],
    "audio": {
      "format": {
        "type": "audio/pcm",
        "rate": 24000
      },
      "output": {
        "voice": "marin"
      }
    },
    "delegation": {
      "type": "client"
    }
  },
  "usage": {
    "seconds": 45.8
  }
}

error

Reports an error in the Live session, such as an invalid client command. Use error.client_event_id, when present, to identify the command that caused the error.

error: Error { code, message, type, 2 more }

Details of the Live error and the client command that caused it, when known.

event_id: string

The unique ID of the Live server event.

type: "error"

The event type, always error.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

OBJECT

error

{
  "type": "error",
  "event_id": "evt_error_001",
  "error": {
    "type": "invalid_request_error",
    "code": "unknown_parameter",
    "message": "Unknown parameter: 'session.voice'.",
    "param": "session.voice",
    "client_event_id": "evt_invalid_001"
  }
}

info

An informational notice about the Live session, such as the event permissions applied to a frontend data channel.

code: string

A machine-readable code for the notice, such as data_channel_permissions.

event_id: string

The unique ID of the Live server event.

message: string

A human-readable explanation of the Live session notice.

type: "info"

The event type, always info.

client_event_id: optional string

The event_id of the client command associated with this server event, when supplied.

session.input_audio.append

Input audio received from the primary transport and reflected to a Live sideband connection before model-input muting.

audio: string

Base64-encoded raw mono PCM16LE at 24 kHz received from the primary transport, reflected to the sideband before model-input muting. This server event uses the same audio key as the client command, but is not an acknowledgment of it.

type: "session.input_audio.append"

The event type, always session.input_audio.append.

transport.dtmf.received

A SIP DTMF keypress received from the caller. Delivered only to sideband observers.

event: string

minLength1

maxLength1

event_id: string

type: "transport.dtmf.received"

transport.dtmf.send

A SIP DTMF keypress successfully sent to the SIP trunk. Delivered only to sideband observers; this is not a client command.

event: string

minLength1

maxLength1

event_id: string

type: "transport.dtmf.send"

client_event_id: optional string

The event_id of the client command, when supplied.

Suggested

Connection

Inputs

Outputs