AI API uptime monitoring: what to probe and when to alert
An AI API can fail in more ways than a web page. It can return 500s. It can return 529 or 503 because the provider is overloaded. It can return 429 for your key while everyone else is fine. It can answer 200 in 40 seconds instead of 2. One model can fail while the others work, and one region can fail while the others answer.
Slow is the hardest of these to call, because the same answer time is fine for one feature and an outage for another:
How slow is too slow? Time until the user sees the first output
Show as a table
| Use | Fine | Users notice | Feels broken |
|---|---|---|---|
| Code completion | under 0.3 s | 0.3–1 s | over 1 s |
| Voice agent, first audio | under 0.8 s | 0.8–2 s | over 2 s |
| Chat, first token | under 1 s | 1–10 s | over 10 s |
| Agent step | under 10 s | 10–30 s | over 30 s |
| Background job | until your timeout | — | timeout |
The provider's status page catches some of these, usually late. Your own traffic catches the rest, but only when you have traffic. A probe from outside covers the gap: a small real request, on a schedule, from more than one place, that tells you whether your key, your model, and your path to the provider work right now.
This post covers what that probe should send, how to read its answer, when to alert, and what it costs.
Three signals, three different questions
| Signal | What it answers | Cost |
|---|---|---|
| The provider's status feed | Has the provider declared an incident? | Nothing |
An authenticated read, such as GET /v1/models |
Is the API reachable, and does my key work? | No tokens |
| A tiny request to the model you ship | Does my model answer, and how fast? | A few tokens per run |
Run all three. The status feed gives you context and wording for your own status page. The read is cheap enough to run often. The model request is the only one that proves the model works, so it is the one that should page someone.
If your app already logs every model call, error rates from that traffic are the best signal during busy hours. Keep the probe anyway: it runs at 3 a.m. when nobody is calling the API, it gives you a clean latency baseline with the same request every time, and it still works when the failure is in your own code path.
The request to send
Send the smallest request your app actually makes, to the model your app actually uses, with a key reserved for monitoring. Keep the prompt short and the output limit as low as the model accepts.
OpenAI
curl -s https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_MONITOR_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"'"$MODEL"'","messages":[{"role":"user","content":"Reply with: ok"}],"max_completion_tokens":16,"reasoning_effort":"none"}'
Newer OpenAI models reject max_tokens and need max_completion_tokens. Reasoning models count their thinking against that limit, and some reject a limit of 1, so leave a little room and set the lowest reasoning_effort the model accepts. Drop reasoning_effort for models that do not reason.
Anthropic
curl -s https://api.anthropic.com/v1/messages \
-H "x-api-key: $ANTHROPIC_MONITOR_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "Content-Type: application/json" \
-d '{"model":"'"$MODEL"'","max_tokens":1,"messages":[{"role":"user","content":"Reply with: ok"}]}'
Google Gemini
curl -s "https://generativelanguage.googleapis.com/v1beta/models/$MODEL:generateContent" \
-H "x-goog-api-key: $GEMINI_MONITOR_KEY" \
-H "Content-Type: application/json" \
-d '{"contents":[{"parts":[{"text":"Reply with: ok"}]}],"generationConfig":{"maxOutputTokens":1}}'
Put the Gemini key in the header, not in a ?key= query parameter. URLs end up in logs and alert messages, where a header does not.
OpenAI-compatible APIs. Groq, Mistral, xAI, DeepSeek, Together, Fireworks, and OpenRouter accept the OpenAI request shape at their own /chat/completions URL with Authorization: Bearer. Check the exact path in each provider's docs: some put it under /openai/v1/ or /inference/v1/.
Check the finish reason, not the text
With a tiny output limit, the reply text is often empty, especially from reasoning models. An assertion like "the response contains ok" fails on a model that is working fine. Assert that the finish-reason field is present instead:
| Provider | Field | Typical value with a tiny limit |
|---|---|---|
| OpenAI and compatible | $.choices[0].finish_reason |
length or stop |
| Anthropic | $.stop_reason |
max_tokens or end_turn |
| Gemini | $.candidates[0].finishReason |
MAX_TOKENS or STOP |
A 200 with that field means the model ran and produced output. A 200 without it is worth alerting on: a 200 can still be down.
The cheap read
For a check that runs often, call the model list with your key. It costs no tokens.
curl -s -o /dev/null -w "%{http_code} %{time_total}s\n" \
https://api.anthropic.com/v1/models \
-H "x-api-key: $ANTHROPIC_MONITOR_KEY" -H "anthropic-version: 2023-06-01"
The same read works at https://api.openai.com/v1/models with Authorization: Bearer and at https://generativelanguage.googleapis.com/v1beta/models with x-goog-api-key. Without a key, OpenAI and Anthropic answer 401 and Gemini answers 403, so a monitor that treats any response below 500 as up will report all three as up with no key at all. Expect exactly 200.
This proves the network path, TLS, the API front door, and your key. It does not prove that any model can serve a request. Model capacity fails separately, and that is the common failure.
What the status code means
| Answer | What it usually means | Alert? |
|---|---|---|
| 200 with a finish reason | Up | No |
| 401 or 403 | Your key was revoked, expired, or lost access to the model | Yes, to you. Users see an outage even though the provider is fine |
| 404 | The model name is wrong or was retired | Yes. Model retirements break apps on a known date |
| 429 | Your rate limit or quota. OpenAI also answers 429 when the account is out of credit | Yes, if it repeats. Check billing before blaming the provider |
| 400 | A bad request body, or on some providers a billing problem such as a low credit balance | Yes. Read the error message |
| 500, 502, 503, 504 | A provider-side failure | Yes, after confirmation |
| 529 (Anthropic) | The API is overloaded | Yes, after confirmation. Retries often succeed within minutes |
| Timeout | The provider or the path to it is too slow to answer | Yes, after confirmation |
Read the error body as well as the code. Providers put the reason in JSON, for example {"type":"error","error":{"type":"authentication_error",...}} from Anthropic, and that text is what tells a billing problem apart from an outage.
Read the status feed too
Most AI providers publish a machine-readable status summary. These answered with JSON on 2026-11-20:
https://status.openai.com/api/v2/status.jsonhttps://status.claude.com/api/v2/status.json(Anthropic)https://status.mistral.ai/api/v2/status.json
Each returns "indicator": none when nothing is declared, then minor, major, or critical. Some feeds are HTML only, such as Gemini's at aistudio.google.com/status, and some sit behind bot protection that blocks plain requests.
A status feed is a lagging signal. A provider posts after it has confirmed a problem, a "partial outage" banner may or may not cover your model, and a quota or key problem never shows up there. Use it to explain an alert, not to raise one.
Latency: decide what slow means
A model that answers in 40 seconds is down for a chat UI and fine for a nightly batch. The chart at the top gives rough ranges per use. For alerts, pick the limit from your own numbers:
- Run the probe for a week and note the typical time and the slowest 5% from each region.
- Set the slow limit at about two to three times that slow-5% figure. Lower makes noise, higher hides real slowdowns.
- Set the timeout well above the slow limit, usually 15 or 30 seconds for a model call, so a slow answer reads as slow, not as down.
- Alert on slow separately from down. Slow usually means degraded capacity, and it often comes before a full outage.
Measure time to first byte as well as total time. For a non-streaming request with a tiny output the two are close. For streaming apps, time to first token is what users feel. A probe with "stream": true and a time-to-first-byte limit approximates it, but it cannot see a stream that stalls halfway through a long answer. That needs measurement in your own app.
The tiny probe also measures the best case. Long prompts, long context, and long outputs take longer, so your app's real latency sits above the probe. Compare like with like: the probe against its own history, not against your production p95.
What probes cost
Requests per month, per region:
| Interval | Requests per region | From 5 regions |
|---|---|---|
| 30 seconds | 86,400 | 432,000 |
| 1 minute | 43,200 | 216,000 |
| 5 minutes | 8,640 | 43,200 |
A request like the ones above uses about 10 to 20 input tokens and 1 to 16 output tokens. At an example price of $1 per million input tokens and $4 per million output tokens, every minute from 5 regions is about 4.3 million input tokens a month, roughly $4, plus $1 to $14 of output depending on the limit. Every 5 minutes from 5 regions is a fifth of that. Small models cost a fraction of it. Check the current price of the model you probe and do the same arithmetic.
Two rules keep the bill and the blast radius small:
- Use a separate key with a hard spending limit. A probe should never be able to run up a large bill, and a revoked monitoring key should not take down production.
- Probe the model you ship, not a cheaper one. A cheap model being up tells you little about the model your users hit. If cost matters, probe the cheap read often and the real model every 5 minutes.
Confirm before you page anyone
Model APIs time out now and then, and a single failed probe is usually noise. One failed check is noise covers this in detail. For AI APIs:
- Wait for a pattern. Alert after 2 or 3 failed runs in a row.
- Check from more than one region. If a provider fails from Frankfurt and works from Virginia, that is a regional or routing problem, and your EU users need to hear about it. If every region fails, it is the provider. Why we ping from different regions explains what each pattern means.
- Alert per provider and per model. One alert for "AI is down" hides which fallback to switch to.
If you route between providers, through a gateway or your own fallback code, probe each provider directly and probe your own endpoint too. The direct probes tell you which provider failed. The probe on your endpoint tells you whether the fallback worked.
What an outside probe does not cover
- Answer quality. A model that answers 200 with wrong or empty output for real prompts looks up. Evaluations and output checks belong in your app.
- Your real latency. The probe measures a tiny request. Long prompts and long outputs need measurement in your own traffic.
- Streams that stall. A probe sees the first bytes. It does not see a stream that stops halfway through.
- Per-customer problems. If each customer brings their own key, one probe with your key cannot see their quota.
- Automatic failover. A probe tells you a provider failed. Switching traffic is your code's or your gateway's job.
A starting list
- A tiny request to each model you ship, with a monitoring key, asserting status 200 and the finish-reason field.
- A
GET /v1/modelsread per provider, expecting exactly 200, on a shorter interval if you want earlier warning of key problems. - A check on each provider's status feed, used to explain alerts.
- A timeout of 15 to 30 seconds, and a separate slow limit at two to three times your measured slow-5% time.
- Alerts after 2 or 3 failures in a row, from at least two regions, per provider and per model.
- A monitoring key with a spending limit, and the cost worked out before you shorten the interval.
The status page tells you what the provider admits. Your own probe tells you whether your app can call the model right now.
Setting it up in QuickStatus.dev
Everything above works with any monitor that can send a POST with secret headers and check the body. In QuickStatus.dev, each probe is one HTTP check.
The model request. Open Checks, click Add check, and pick Website or API. Enter the provider URL and set Up when to Status is 200 OK. Under HTTP request, set the method to POST, add the auth header and Content-Type: application/json under Request headers, and paste the request body. Under Assertions, add Body contains with the finish-reason field name: finish_reason, stop_reason, or finishReason. It is an exact, case-sensitive text match. Monitor the OpenAI API walks through the same steps for OpenAI.
The cheap read and the status feed. The /v1/models read is the same check with method GET and no body. For a status feed, point an HTTP check at its status.json URL and add JSONPath equals with $.status.indicator equal to none. The check then goes down when the provider declares an incident, and you can link it to its own status page component.
When to alert. Under When to alert, set Mark down after to 2 or 3 and Request timeout to 15 or 30 seconds. Under Slow responses, set Slow above (ms) and measure on Total time or Time to first byte. A slow run marks the check Slow, not down, sends an alert, and leaves uptime alone.
Regions and intervals. Free checks every 5 minutes from 1 region. Pro checks every 60 seconds from up to five regions: San Francisco, Ashburn, Montreal, Frankfurt, and Singapore. Scale checks every 30 seconds. Every run stores its duration and region, so the check page shows whether a slowdown is everywhere or in one place.
What failed. When a run fails, the check's results show what the provider answered, such as authentication_error or an overloaded message, with header values replaced by [redacted]. Alerts carry the status code. Email and Telegram alerts are on every plan. Slack, Discord, PagerDuty, Pushover, and webhook alerts are Pro.
Status page. Link each check to a component, for example Chat (OpenAI) or Summaries (Anthropic), so a provider outage shows as one feature degraded rather than everything down. Under When the check is down, choose Create an incident automatically or Ask me to review first.
How the providers compare. LLM API uptime rankings runs this probe, a tiny request with a finish-reason check, against OpenAI, Anthropic, Gemini, xAI, Mistral, DeepSeek, Cohere, and Moonshot every 5 minutes from all five regions, and publishes weekly uptime.
Limits. Checks only reach public URLs, so a model server inside your VPC needs a public health route. A check measures one request: it does not follow a stream to the end, score answers, or fail traffic over to another provider. Auth headers and request bodies are set in the console. The uptime monitoring MCP server can create, run, and link these checks from Claude Code, Cursor, or Codex, but it leaves secrets in the console.
When the model API your product depends on fails, your users should hear it from you before they open a ticket. Start free with 10 checks and a status page, no card. See pricing.