Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
968 changes: 484 additions & 484 deletions docs/en/.source-manifest.json

Large diffs are not rendered by default.

10 changes: 10 additions & 0 deletions docs/en/api/docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@

Upcoming deprecations are listed on the [deprecations page](/api/docs/deprecations).

## September, 2026

### Sep 2

Update

Updated API errors so applications can distinguish traffic that increases too quickly from temporary model overload.

Traffic that increases too quickly can return a `429` error with the `slow_down` code. Temporary model overload returns a `503` error with the `server_is_overloaded` code. Both responses may include `Retry-After`. When the header is present, wait at least as long as it specifies before retrying. If it's missing, use exponential backoff. See the [error codes guide](https://developers.openai.com/api/docs/guides/error-codes) and [rate limits guide](https://developers.openai.com/api/docs/guides/rate-limits).

## August, 2026

### Aug 29
Expand Down
2 changes: 1 addition & 1 deletion docs/en/api/docs/guides/background.md
Original file line number Diff line number Diff line change
Expand Up @@ -687,7 +687,7 @@ last_sequence_number = -1
response_id = ""
stream.each do |event|
puts(event.type)
last_sequence_number = event.sequence_number
last_sequence_number = event.sequence_number || last_sequence_number
if event.is_a?(OpenAI::Models::Responses::ResponseCreatedEvent)
response_id = event.response.id
end
Expand Down
76 changes: 36 additions & 40 deletions docs/en/api/docs/guides/error-codes.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/en/api/docs/guides/fast-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ Fast mode consumption counts toward rate limits the same way as Standard process

**Ramp rate limit**

If your traffic ramps too fast, the system may downgrade some Fast mode requests to standard speeds and charge standard rates. When this happens, the response contains `service_tier: "default"`. The ramp rate limit may apply if you send at least 1 million tokens per minute (TPM) and increase TPM by more than 50% within 15 minutes.
If your traffic ramps too fast, the system may downgrade some Fast mode requests to standard speeds and charge standard rates. When this happens, the response contains `service_tier: "default"`. As a rule of thumb, once your traffic reaches 1 million input tokens per minute (TPM), increase it by no more than 50% every 15 minutes. The exact point at which the ramp-rate limit applies can vary by model and traffic conditions.

To avoid triggering the ramp rate limit:

Expand Down
7 changes: 5 additions & 2 deletions docs/en/api/docs/guides/function-calling.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,8 @@ func main() {
if err := json.Unmarshal([]byte(call.Arguments), &arguments); err != nil {
panic(err)
}
functionOutput = responses.ResponseInputItemParamOfFunctionCallOutput(call.CallID, getHoroscope(arguments.Sign))
functionOutput = responses.ResponseInputItemParamOfFunctionCallOutput(getHoroscope(arguments.Sign))
functionOutput.OfFunctionCallOutput.CallID = openai.String(call.CallID)
}
if functionOutput.OfFunctionCallOutput == nil {
panic("the model did not call get_horoscope")
Expand Down Expand Up @@ -680,7 +681,9 @@ for _, output := range response.Output {
if err != nil {
panic(err)
}
input = append(input, responses.ResponseInputItemParamOfFunctionCallOutput(toolCall.CallID, result))
toolOutput := responses.ResponseInputItemParamOfFunctionCallOutput(result)
toolOutput.OfFunctionCallOutput.CallID = openai.String(toolCall.CallID)
input = append(input, toolOutput)
}
```

Expand Down
42 changes: 23 additions & 19 deletions docs/en/api/docs/guides/moderation.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,27 +153,31 @@ ResponseCreateParams params =
.build();

var response = client.responses().create(params);
JsonValue moderation = response._additionalProperties().get("moderation");
if (moderation == null) {
throw new IllegalStateException("The response did not include moderation results");
}
Map<?, ?> results = moderation.convert(Map.class);
var moderation =
response
.moderation()
.orElseThrow(
() -> new IllegalStateException("The response did not include moderation results"));
List<Boolean> flags = new ArrayList<>();
for (String side : List.of("input", "output")) {
if (!(results.get(side) instanceof Map<?, ?> result)) {
throw new IllegalStateException("Missing " + side + " moderation result");
}
if ("error".equals(result.get("type"))) {
throw new IllegalStateException(String.valueOf(result.get("message")));
}
if (!"moderation_result".equals(result.get("type"))) {
throw new IllegalStateException("Unexpected " + side + " moderation result type");
}
if (!(result.get("flagged") instanceof Boolean flagged)) {
throw new IllegalStateException("Missing " + side + " moderation flag");
}
flags.add(flagged);

var input = moderation.input();
if (input.isError()) {
throw new IllegalStateException(input.asError().message());
}
if (!input.isModerationResult()) {
throw new IllegalStateException("Missing input moderation flag");
}
flags.add(input.asModerationResult().flagged());

var output = moderation.output();
if (output.isError()) {
throw new IllegalStateException(output.asError().message());
}
if (!output.isModerationResult()) {
throw new IllegalStateException("Missing output moderation flag");
}
flags.add(output.asModerationResult().flagged());

flags.forEach(System.out::println);
```

Expand Down
17 changes: 17 additions & 0 deletions docs/en/api/docs/guides/rate-limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,23 @@ curl https://api.openai.com/v1/fine_tuning/model_limits \

## Error mitigation

### Handle rapid traffic increases and model overload

The API can return `slow_down` when your request rate increases too quickly, or `server_is_overloaded` when the requested model is temporarily overloaded. Check the HTTP status and `error.code` to tell these conditions apart:

| HTTP status | Error type | Error code | What it means | What to do |
| ----------- | --------------------------- | ---------------------- | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `429` | `rate_limit_error` | `slow_down` | Your request rate increased too quickly. | Follow `Retry-After` when it's present, reduce your request rate, and then increase it gradually. |
| `503` | `service_unavailable_error` | `server_is_overloaded` | The requested model is temporarily overloaded. | Follow `Retry-After` when it's present, then retry. If the error continues, increase the delay between retry attempts. |

If `Retry-After` is missing, increase the delay between retries and add a small random delay.

A `slow_down` error can occur even when your traffic is within its requests-per-minute and tokens-per-minute limits. It reflects how quickly traffic increased, not whether you exhausted those limits.

As a rule of thumb, once your traffic reaches 1 million input tokens per minute (TPM), increase it by no more than 50% every 15 minutes. The exact point at which the ramp-rate limit applies can vary by model and traffic conditions.

Enterprise customers whose pay-as-you-go traffic routinely hits ramp-rate limits can consider [Scale Tier](https://openai.com/api-scale-tier/) for more predictable capacity on eligible models. For GPT-5.6 and later models, see [Reserved Tier](https://openai.com/api-reserved-tier/). Capacity tiers don't change how you should handle a `slow_down` response: follow `Retry-After` when it's present, reduce traffic, and ramp gradually.

### What are some steps I can take to mitigate this?

The OpenAI Cookbook has a [Python notebook](https://developers.openai.com/cookbook/examples/how_to_handle_rate_limits) that explains how to avoid rate limit errors, as well an example [Python script](https://github.com/openai/openai-cookbook/blob/main/examples/api_request_parallel_processor.py) for staying under rate limits while batch processing API requests.
Expand Down
Loading