Skip to content
QuickStatus.dev

Blog

AI API uptime monitoring: what to probe and when to alert

Updated on by Nurbol Sakenov

An AI API can fail in more ways than a web page. It can return 500s. It can return 529 or 503 because the provider is overloaded. It can return 429 for your key while everyone else is fine. It can answer 200 in 40 seconds instead of 2. One model can fail while the others work, and one region can fail while the others answer.

Slow is the hardest of these to call, because the same answer time is fine for one feature and an outage for another:

How slow is too slow? Time until the user sees the first output

Code completionfine under 0.3 s, broken past 1 s
Voice agent, first audiofine under 0.8 s, broken past 2 s
Chat, first tokenfine under 1 s, broken past 10 s
Agent stepfine under 10 s, broken past 30 s
Background jobfine until your timeout
Show as a table
UseFineUsers noticeFeels broken
Code completionunder 0.3 s0.3–1 sover 1 s
Voice agent, first audiounder 0.8 s0.8–2 sover 2 s
Chat, first tokenunder 1 s1–10 sover 10 s
Agent stepunder 10 s10–30 sover 30 s
Background jobuntil your timeout—timeout
Rules of thumb for the time until a user sees the first output, not measurements of any provider. The 0.1, 1, and 10 second marks follow Jakob Nielsen's response time limits. Log scale. Measure your own baseline before you set alert limits.

The provider's status page catches some of these, usually late. Your own traffic catches the rest, but only when you have traffic. A probe from outside covers the gap: a small real request, on a schedule, from more than one place, that tells you whether your key, your model, and your path to the provider work right now.

This post covers what that probe should send, how to read its answer, when to alert, and what it costs.

Three signals, three different questions

Signal What it answers Cost
The provider's status feed Has the provider declared an incident? Nothing
An authenticated read, such as GET /v1/models Is the API reachable, and does my key work? No tokens
A tiny request to the model you ship Does my model answer, and how fast? A few tokens per run
How far each AI API check reaches The status feed check reads the provider's status page, which staff update after an incident is confirmed. The cheap read, GET /v1/models, reaches the API front door and proves the network, TLS, and your key. Only the tiny model request goes through the front door to the model and proves it can answer. Status page what the provider declared Provider API Front door network, TLS, your key Model capacity to answer posted later Status feed status.json Cheap read GET /v1/models Model request POST, tiny limit How far each AI API check reaches The cheap read, GET /v1/models, stops at the API front door and proves the network, TLS, and your key. The tiny model request goes through the front door to the model and proves it can answer. The status feed check reads the provider's status page, which is posted after an incident is confirmed. Cheap read GET /v1/models Model request POST, tiny limit Provider API Front door network, TLS, your key Model capacity to answer posted later Status page what the provider declared Status feed status.json
Each check reaches one layer further. The status page is updated by people after an incident is confirmed, and the cheap read stops at the front door. Only the model request proves the model can answer, so it is the one to alert on.

Run all three. The status feed gives you context and wording for your own status page. The read is cheap enough to run often. The model request is the only one that proves the model works, so it is the one that should page someone.

If your app already logs every model call, error rates from that traffic are the best signal during busy hours. Keep the probe anyway: it runs at 3 a.m. when nobody is calling the API, it gives you a clean latency baseline with the same request every time, and it still works when the failure is in your own code path.

The request to send

Send the smallest request your app actually makes, to the model your app actually uses, with a key reserved for monitoring. Keep the prompt short and the output limit as low as the model accepts.

OpenAI

curl -s https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_MONITOR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"'"$MODEL"'","messages":[{"role":"user","content":"Reply with: ok"}],"max_completion_tokens":16,"reasoning_effort":"none"}'

Newer OpenAI models reject max_tokens and need max_completion_tokens. Reasoning models count their thinking against that limit, and some reject a limit of 1, so leave a little room and set the lowest reasoning_effort the model accepts. Drop reasoning_effort for models that do not reason.

Anthropic

curl -s https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_MONITOR_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"'"$MODEL"'","max_tokens":1,"messages":[{"role":"user","content":"Reply with: ok"}]}'

Google Gemini

curl -s "https://generativelanguage.googleapis.com/v1beta/models/$MODEL:generateContent" \
  -H "x-goog-api-key: $GEMINI_MONITOR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"contents":[{"parts":[{"text":"Reply with: ok"}]}],"generationConfig":{"maxOutputTokens":1}}'

Put the Gemini key in the header, not in a ?key= query parameter. URLs end up in logs and alert messages, where a header does not.

OpenAI-compatible APIs. Groq, Mistral, xAI, DeepSeek, Together, Fireworks, and OpenRouter accept the OpenAI request shape at their own /chat/completions URL with Authorization: Bearer. Check the exact path in each provider's docs: some put it under /openai/v1/ or /inference/v1/.

Check the finish reason, not the text

With a tiny output limit, the reply text is often empty, especially from reasoning models. An assertion like "the response contains ok" fails on a model that is working fine. Assert that the finish-reason field is present instead:

Provider Field Typical value with a tiny limit
OpenAI and compatible $.choices[0].finish_reason length or stop
Anthropic $.stop_reason max_tokens or end_turn
Gemini $.candidates[0].finishReason MAX_TOKENS or STOP

A 200 with that field means the model ran and produced output. A 200 without it is worth alerting on: a 200 can still be down.

The cheap read

For a check that runs often, call the model list with your key. It costs no tokens.

curl -s -o /dev/null -w "%{http_code} %{time_total}s\n" \
  https://api.anthropic.com/v1/models \
  -H "x-api-key: $ANTHROPIC_MONITOR_KEY" -H "anthropic-version: 2023-06-01"

The same read works at https://api.openai.com/v1/models with Authorization: Bearer and at https://generativelanguage.googleapis.com/v1beta/models with x-goog-api-key. Without a key, OpenAI and Anthropic answer 401 and Gemini answers 403, so a monitor that treats any response below 500 as up will report all three as up with no key at all. Expect exactly 200.

This proves the network path, TLS, the API front door, and your key. It does not prove that any model can serve a request. Model capacity fails separately, and that is the common failure.

What the status code means

Answer What it usually means Alert?
200 with a finish reason Up No
401 or 403 Your key was revoked, expired, or lost access to the model Yes, to you. Users see an outage even though the provider is fine
404 The model name is wrong or was retired Yes. Model retirements break apps on a known date
429 Your rate limit or quota. OpenAI also answers 429 when the account is out of credit Yes, if it repeats. Check billing before blaming the provider
400 A bad request body, or on some providers a billing problem such as a low credit balance Yes. Read the error message
500, 502, 503, 504 A provider-side failure Yes, after confirmation
529 (Anthropic) The API is overloaded Yes, after confirmation. Retries often succeed within minutes
Timeout The provider or the path to it is too slow to answer Yes, after confirmation

Read the error body as well as the code. Providers put the reason in JSON, for example {"type":"error","error":{"type":"authentication_error",...}} from Anthropic, and that text is what tells a billing problem apart from an outage.

Read the status feed too

Most AI providers publish a machine-readable status summary. These answered with JSON on 2026-11-20:

  • https://status.openai.com/api/v2/status.json
  • https://status.claude.com/api/v2/status.json (Anthropic)
  • https://status.mistral.ai/api/v2/status.json

Each returns "indicator": none when nothing is declared, then minor, major, or critical. Some feeds are HTML only, such as Gemini's at aistudio.google.com/status, and some sit behind bot protection that blocks plain requests.

A status feed is a lagging signal. A provider posts after it has confirmed a problem, a "partial outage" banner may or may not cover your model, and a quota or key problem never shows up there. Use it to explain an alert, not to raise one.

Latency: decide what slow means

A model that answers in 40 seconds is down for a chat UI and fine for a nightly batch. The chart at the top gives rough ranges per use. For alerts, pick the limit from your own numbers:

  1. Run the probe for a week and note the typical time and the slowest 5% from each region.
  2. Set the slow limit at about two to three times that slow-5% figure. Lower makes noise, higher hides real slowdowns.
  3. Set the timeout well above the slow limit, usually 15 or 30 seconds for a model call, so a slow answer reads as slow, not as down.
  4. Alert on slow separately from down. Slow usually means degraded capacity, and it often comes before a full outage.

Measure time to first byte as well as total time. For a non-streaming request with a tiny output the two are close. For streaming apps, time to first token is what users feel. A probe with "stream": true and a time-to-first-byte limit approximates it, but it cannot see a stream that stalls halfway through a long answer. That needs measurement in your own app.

The tiny probe also measures the best case. Long prompts, long context, and long outputs take longer, so your app's real latency sits above the probe. Compare like with like: the probe against its own history, not against your production p95.

What probes cost

Requests per month, per region:

Interval Requests per region From 5 regions
30 seconds 86,400 432,000
1 minute 43,200 216,000
5 minutes 8,640 43,200

A request like the ones above uses about 10 to 20 input tokens and 1 to 16 output tokens. At an example price of $1 per million input tokens and $4 per million output tokens, every minute from 5 regions is about 4.3 million input tokens a month, roughly $4, plus $1 to $14 of output depending on the limit. Every 5 minutes from 5 regions is a fifth of that. Small models cost a fraction of it. Check the current price of the model you probe and do the same arithmetic.

Two rules keep the bill and the blast radius small:

  • Use a separate key with a hard spending limit. A probe should never be able to run up a large bill, and a revoked monitoring key should not take down production.
  • Probe the model you ship, not a cheaper one. A cheap model being up tells you little about the model your users hit. If cost matters, probe the cheap read often and the real model every 5 minutes.

Confirm before you page anyone

Model APIs time out now and then, and a single failed probe is usually noise. One failed check is noise covers this in detail. For AI APIs:

  • Wait for a pattern. Alert after 2 or 3 failed runs in a row.
  • Check from more than one region. If a provider fails from Frankfurt and works from Virginia, that is a regional or routing problem, and your EU users need to hear about it. If every region fails, it is the provider. Why we ping from different regions explains what each pattern means.
  • Alert per provider and per model. One alert for "AI is down" hides which fallback to switch to.

If you route between providers, through a gateway or your own fallback code, probe each provider directly and probe your own endpoint too. The direct probes tell you which provider failed. The probe on your endpoint tells you whether the fallback worked.

What an outside probe does not cover

  • Answer quality. A model that answers 200 with wrong or empty output for real prompts looks up. Evaluations and output checks belong in your app.
  • Your real latency. The probe measures a tiny request. Long prompts and long outputs need measurement in your own traffic.
  • Streams that stall. A probe sees the first bytes. It does not see a stream that stops halfway through.
  • Per-customer problems. If each customer brings their own key, one probe with your key cannot see their quota.
  • Automatic failover. A probe tells you a provider failed. Switching traffic is your code's or your gateway's job.

A starting list

  1. A tiny request to each model you ship, with a monitoring key, asserting status 200 and the finish-reason field.
  2. A GET /v1/models read per provider, expecting exactly 200, on a shorter interval if you want earlier warning of key problems.
  3. A check on each provider's status feed, used to explain alerts.
  4. A timeout of 15 to 30 seconds, and a separate slow limit at two to three times your measured slow-5% time.
  5. Alerts after 2 or 3 failures in a row, from at least two regions, per provider and per model.
  6. A monitoring key with a spending limit, and the cost worked out before you shorten the interval.

The status page tells you what the provider admits. Your own probe tells you whether your app can call the model right now.

Setting it up in QuickStatus.dev

Everything above works with any monitor that can send a POST with secret headers and check the body. In QuickStatus.dev, each probe is one HTTP check.

The model request. Open Checks, click Add check, and pick Website or API. Enter the provider URL and set Up when to Status is 200 OK. Under HTTP request, set the method to POST, add the auth header and Content-Type: application/json under Request headers, and paste the request body. Under Assertions, add Body contains with the finish-reason field name: finish_reason, stop_reason, or finishReason. It is an exact, case-sensitive text match. Monitor the OpenAI API walks through the same steps for OpenAI.

The cheap read and the status feed. The /v1/models read is the same check with method GET and no body. For a status feed, point an HTTP check at its status.json URL and add JSONPath equals with $.status.indicator equal to none. The check then goes down when the provider declares an incident, and you can link it to its own status page component.

When to alert. Under When to alert, set Mark down after to 2 or 3 and Request timeout to 15 or 30 seconds. Under Slow responses, set Slow above (ms) and measure on Total time or Time to first byte. A slow run marks the check Slow, not down, sends an alert, and leaves uptime alone.

Regions and intervals. Free checks every 5 minutes from 1 region. Pro checks every 60 seconds from up to five regions: San Francisco, Ashburn, Montreal, Frankfurt, and Singapore. Scale checks every 30 seconds. Every run stores its duration and region, so the check page shows whether a slowdown is everywhere or in one place.

What failed. When a run fails, the check's results show what the provider answered, such as authentication_error or an overloaded message, with header values replaced by [redacted]. Alerts carry the status code. Email and Telegram alerts are on every plan. Slack, Discord, PagerDuty, Pushover, and webhook alerts are Pro.

Status page. Link each check to a component, for example Chat (OpenAI) or Summaries (Anthropic), so a provider outage shows as one feature degraded rather than everything down. Under When the check is down, choose Create an incident automatically or Ask me to review first.

How the providers compare. LLM API uptime rankings runs this probe, a tiny request with a finish-reason check, against OpenAI, Anthropic, Gemini, xAI, Mistral, DeepSeek, Cohere, and Moonshot every 5 minutes from all five regions, and publishes weekly uptime.

Limits. Checks only reach public URLs, so a model server inside your VPC needs a public health route. A check measures one request: it does not follow a stream to the end, score answers, or fail traffic over to another provider. Auth headers and request bodies are set in the console. The uptime monitoring MCP server can create, run, and link these checks from Claude Code, Cursor, or Codex, but it leaves secrets in the console.

When the model API your product depends on fails, your users should hear it from you before they open a ticket. Start free with 10 checks and a status page, no card. See pricing.

Validation report

Validation summary

Status validated

Date: 2026-11-20

What was checked

Request shapes were compared with the /uptime-rankings/ probe code. Public endpoints were requested live. Every QuickStatus.dev claim was compared with the docs, the console source, and /pricing/.

Result

  • Request shapes: OpenAI max_completion_tokens (GPT-5.4 Nano needs 16 with reasoning_effort: "none"), Anthropic x-api-key + anthropic-version: 2023-06-01 + max_tokens: 1, Gemini x-goog-api-key + generationConfig.maxOutputTokens: 1. Source: data_pipeline/engine/domain/rankings_probes.py, data_pipeline/engine/rankings/catalog.json, docs/roadmap/2026-11-17-ai-api-uptime-rankings.md (verified with our keys 2026-11-18).
  • Finish-reason fields: $.choices[0].finish_reason, $.stop_reason, $.candidates[0].finishReason; reply text is often empty at a tiny limit. Source: same roadmap doc and rankings_probes.py.
  • OpenAI-compatible paths: Groq /openai/v1/, Fireworks /inference/v1/. Source: catalog.json.
  • Unauthenticated reads (2026-11-20, curl): GET api.openai.com/v1/models 401 "Missing bearer authentication in header"; GET api.anthropic.com/v1/models 401 authentication_error "x-api-key header is required"; GET generativelanguage.googleapis.com/v1beta/models 403 PERMISSION_DENIED.
  • Status feeds (2026-11-20, curl): status.openai.com/api/v2/status.json 200 JSON, indicator none; status.claude.com/api/v2/status.json 200 JSON, indicator none; status.mistral.ai/api/v2/status.json 200 JSON, indicator minor; status.x.ai/api/v2/status.json 403 HTML (bot protection); aistudio.google.com/status 200 HTML only.
  • Cost math: 43,200 runs/month per region at 1 minute; 216,000 from 5 regions; 20 input tokens × 216,000 = 4.32 M (≈ $4.32 at $1/M); output 1–16 tokens × 216,000 at $4/M = $0.86–$13.82. Prices are labelled as an example.
  • Checks: Up when Status is 200 OK; HTTP request method, headers, body; Body contains is exact and case-sensitive; JSONPath equals compares the string value, so $.status.indicator = none matches the feeds above. Source: content/docs/6-checks.md, data_pipeline/engine/workers/checkers/http.py.
  • When to alert: Mark down after 1/2/3/5 (default 3); Request timeout 5/10/15/30 s; Slow above (ms) on Total time or Time to first byte marks Slow, not down. Source: content/docs/6-checks.md.
  • Failure details: results show the provider's error message, header values replaced with [redacted]; alerts and MCP show the status code alone. Source: content/docs/7-check-results.md.
  • Plans: Free 10 checks, 5 minutes, 1 region; Pro 60 seconds, 5 regions; Scale 30 seconds. Email and Telegram on every plan; Slack, Discord, PagerDuty, Pushover, webhooks on Pro. Source: public/pricing.md, content/docs/6-checks.md.
  • MCP: request headers, request body, and HTTP auth stay in the console. Source: data_pipeline/engine/api/mcp.py.
  • Rankings: 8 LLM providers, every 5 minutes from 5 regions. Source: public/uptime-rankings/llm-apis/.
  • Latency chart: the ranges are labelled as rules of thumb, not measurements. The 0.1/1/10 s marks follow Nielsen's response time limits (https://www.nngroup.com/articles/response-times-3-important-limits/, HTTP 200 on 2026-11-20). Band colors are one-hue ink ramps that pass the dataviz ordinal checks in both themes; identity is also carried by the legend, row text, and a table view.

Not checked

  • Authenticated completions were not sent from this session; the request shapes rely on the 2026-11-18 rankings verification.
  • Anthropic 529 overloaded_error, OpenAI 429 insufficient_quota, and 400 low-credit errors are from provider docs, not reproduced.
  • The per-use thresholds in the latency chart (code completion, voice, chat, agent step) are editorial judgment, not cited studies.
  • Typical finish-reason values (length, max_tokens, MAX_TOKENS) depend on the model.