A Machine for Turning Untrusted Input Into Trusted State

It is tempting to think of a web API as a collection of features. This endpoint lists projects, that one uploads a transcript, this other one kicks off an LLM summarisation run. Organised that way, an API looks like a menu, and keeping it healthy looks like keeping each dish correct.

A more useful description is colder. A web API is a machine for turning untrusted input into trusted state. Something arrives from outside your control — a browser, a survey vendor’s webhook, a scheduled job, someone with a terminal and curiosity — and by the end of the request that input has become a row in your database, a file in a bucket, a charge on a card, or a number on a dashboard that a client will make decisions from. Everything the API does is that conversion. Every endpoint is one more place the conversion can go wrong.

Look at production incidents with that lens and they stop being a long tail of unrelated mysteries. They cluster. Almost all of them are boundary failures: something crossed a joint between two parts of the system while carrying an assumption that was not true on the other side. There are only about five of these mistakes, and they explain most outages. The way to see them is to follow a single request all the way through.

The journey of one request

A researcher at a client clicks “Export responses” on your React dashboard. Here is what that click actually traverses.

1. The browser. It resolves DNS, opens a TLS connection, and sends a POST with a JSON body and a cookie or bearer token. It also starts its own clock: fetch will hang as long as you let it, and the user will reload long before it gives up, sending the whole thing a second time. Your API does not know it is a repeat.

2. A reverse proxy. On Azure App Service or Container Apps there is a front end in front of your process, terminating TLS and forwarding the request. It has opinions you did not set: a maximum body size, and an idle timeout in the low minutes. If your handler takes longer than that, the proxy hangs up on the client and returns a 502 or 504 — while your code carries happily on, finishing work whose result nobody will ever read. This is why “it worked locally” and “it works with small exports” are the two most common bug reports in this class.

3. A uvicorn worker. Your Python process runs an event loop: a single thread that juggles many requests by switching between them whenever one is waiting on the network. That is what makes an async server efficient, and it is also its one fatal weakness. The loop can only switch when your code gives it the chance. Any code that occupies that thread without awaiting stops every request in the process, not only its own.

4. Validation. FastAPI parses the body and hands it to Pydantic, which coerces and checks it against your model. This is the formal boundary between outside and inside: the one line in the system where input officially becomes trusted. How good that boundary is, is how good your API is.

5. Business logic. The part everyone thinks of as the code. Usually the least dangerous part.

6. A database transaction. Reads and writes to Postgres, wrapped — you hope — in a unit of work that either lands completely or not at all.

7. The response. Serialised back to JSON and returned. Which means it is also an outbound boundary, and outbound boundaries leak.

Seven steps, six joints. Now the failures.

Boundary failure 1: no timeout

Every network call your API makes is a promise that something else will answer. Nothing enforces that promise unless you do, and most HTTP clients wait forever by default.

This is the single most common cause of a production API that is “down” while every component reports itself healthy. A survey vendor’s endpoint gets slow — not broken, slow. Your export handler calls it and waits. A second request arrives and waits. Within a minute every worker is holding a dead conversation, and your dashboard, your login endpoint and your health check are all unreachable, because there is nobody left to answer them.

The rule is: no outbound call without a ceiling on how long you will wait. One argument, everywhere, no exceptions.

# One client, created once, with a timeout that applies to every call through it.
client = httpx.AsyncClient(timeout=httpx.Timeout(10.0, connect=5.0))

async def fetch_responses(survey_id: str) -> list[dict]:
    r = await client.get(f"{SURVEY_API}/surveys/{survey_id}/responses")
    r.raise_for_status()                      # a 500 upstream becomes an exception here
    return r.json().get("items", [])

Choose the number deliberately, and make it smaller than the proxy’s idle timeout. A call that takes longer than your caller is willing to wait has no reason to still be running.

Boundary failure 2: a blocking call in async code

This is the failure you can spot by eye, and the one agents introduce most often, because the blocking version of nearly every library is the more famous one.

@app.get("/exports/{export_id}")
async def get_export(export_id: str):
    r = requests.get(f"{SURVEY_API}/exports/{export_id}")   # blocks the whole event loop
    df = pd.read_parquet(local_path)                        # so does this, for seconds
    return {"rows": len(df)}

Both of those lines are ordinary, correct Python. Inside async def they are a denial-of-service attack on your own service: while either runs, the single thread cannot switch, so every other request in that worker is frozen. The tell is visual and worth memorising — an async def handler containing a call with no await in front of it that is clearly doing input or output: requests.get, time.sleep, open(...).read() on a large file, a synchronous database driver, pandas parsing a big file, any crypto or PDF work.

There are two fixes, and choosing between them is the real skill.

@app.get("/exports/{export_id}")
async def get_export(export_id: str):
    r = await client.get(f"{SURVEY_API}/exports/{export_id}")   # async library: await it
    r.raise_for_status()
    # CPU-bound or stubbornly synchronous work: push it off the loop thread
    df = await anyio.to_thread.run_sync(pd.read_parquet, local_path)
    return {"rows": len(df)}

If an async version of the library exists, use it and await it. If it does not — pandas, a PDF parser, an old SDK — move it to a thread. And if the work takes minutes rather than milliseconds, neither fix is right: it does not belong in a request at all. It belongs in a background job that returns an id immediately, which the dashboard can poll. A request-response cycle is the wrong shape for anything a human will not wait for.

Boundary failure 3: over-trusting input

Validation is where an agent’s code is most often technically present and practically useless. A model that accepts a string for every field passes review, runs fine, and defends nothing.

class ExportRequest(BaseModel):
    survey_id: str
    limit: int = 1000
    columns: list[str] = []

Read that as an attacker, or as a tired colleague. limit accepts 50,000,000, so one request can try to hold your entire response table in memory. columns accepts anything, and if it is interpolated into SQL anywhere downstream, you have handed over the database. And there is no client identifier at all, which means the answer to “whose data can this return?” lives somewhere else entirely — in code you did not read.

A boundary that earns its name is narrow on purpose:

class ExportRequest(BaseModel):
    model_config = ConfigDict(extra="forbid")      # reject unknown fields outright
    survey_id: UUID                                # not "any string"
    limit: int = Field(default=1000, ge=1, le=5000)  # a ceiling, enforced before your code runs
    columns: list[Literal["respondent_id", "answered_at", "score"]] = []

@router.post("/exports")
async def create_export(
    body: ExportRequest,
    caller: Annotated[User, Depends(current_user)],   # identity from the token, never the body
) -> ExportAccepted:
    ...

Two ideas do most of the work. First, let the types say no: a UUID, a bounded int, a Literal list of allowed columns and extra="forbid" mean whole classes of bad request never reach your logic. Second, and this is the one to check in your own code today: identity must come from the credential, not the payload. Any endpoint that takes a client_id from the request body and trusts it has no authorisation at all, only a suggestion. The caller controls the body. They do not control your token verification.

Boundary failure 4: the missing transaction boundary

Databases offer a genuinely remarkable guarantee: a group of writes can be made to land all together or not at all. It is yours for free, and it is lost the moment code commits as it goes.

# Three commits: three chances to stop halfway and leave incoherent data behind.
session.add(export); await session.commit()
for row in rows:
    session.add(ResponseRow(**row)); await session.commit()
export.status = "done"; await session.commit()

If the process is recycled during that loop — a deploy, a restart, an out-of-memory kill, all of which happen without warning — you are left with an export marked as running, holding 4,000 of 9,000 rows, and no way for the system to tell that state from a job still in progress. Nobody gets an error. The dashboard shows a number that is quietly wrong, which is worse than a number that is missing.

The transaction boundary is a decision, not a detail: which set of writes must be true together? Write that set inside one block and commit once.

async with session.begin():           # commits on success, rolls back on any exception
    export = Export(survey_id=body.survey_id, status="done")
    session.add(export)
    session.add_all(ResponseRow(export_id=export.id, **r) for r in rows)
# One outcome: everything landed, or nothing did.

Two related smells while you are in there. A transaction held open across a slow network call — a database transaction waiting on an LLM — holds locks for seconds and will eventually deadlock under load; do the slow call first, then open the transaction. And a job with no idempotency key will double-write when the user reloads, because as far as your API is concerned that reload is a brand new and perfectly valid request.

Boundary failure 5: leaking internals on the way out

The response is a boundary too, and it faces outward. Two versions of the same mistake.

The first is the error handler an agent writes to be helpful:

except Exception as e:
    raise HTTPException(status_code=500, detail=str(e))   # ships your internals to the client

The string of a database exception can contain the failing SQL, table and column names, a connection string fragment, or a respondent’s email from a uniqueness violation. It goes into the browser, into the client’s console, into anything that logs responses. Log the detail on your side with an identifier; return something deliberately boring.

except Exception:
    ref = uuid4().hex[:8]
    logger.exception("export failed", extra={"ref": ref})    # full detail stays in your logs
    raise HTTPException(status_code=500, detail=f"Export failed (ref {ref})")

The second version is quieter and more common: returning a database object directly. A handler that returns an ORM row returns every column on it — the internal cost note, the flag marking a respondent as suspect, the soft-delete marker, a column somebody adds next month without thinking about who reads it. Declaring a response model closes the boundary, and keeps it closed as the table grows:

class ExportPublic(BaseModel):
    model_config = ConfigDict(from_attributes=True)
    id: UUID
    status: Literal["queued", "running", "done", "failed"]
    row_count: int
    # Whatever else lives on the table, this is all that leaves the building.

@router.get("/exports/{export_id}")
async def read_export(export_id: UUID, ...) -> ExportPublic:
    ...

What this buys you

The five failures are worth memorising as a set, because together they form a review checklist that takes about a minute per endpoint, needs no deep understanding of the code, and catches an unreasonable share of real incidents:

1. Does every outbound call have a timeout, smaller than the proxy’s?
2. Is there any input or output in an async def without an await?
3. Can the request ask for an unbounded amount of work, and does identity come from the token rather than the body?
4. Which writes must be true together, and are they in one transaction?
5. What exactly leaves in the response, including on the error path?

Notice that not one of those questions is about business logic. They are all questions about joints — the places where a browser meets a proxy, a proxy meets a worker, untrusted text meets a type, a set of writes meets the possibility of being interrupted, and your internals meet somebody else’s screen. That is where the machine does its real work, and it is where the machine breaks.