27Async/await — an event loop over suspended frames

In Chapter 26 we gave one process many threads, and the GIL let only one of them run Python at a time. That is real concurrency for waiting, never for computing. This chapter asks the sharper question hiding underneath. If a thread that is only waiting does no work, why does it cost so much to keep around? A thread blocked on a socket holds an entire 8-megabyte call stack and a slot in the kernel scheduler, all to remember one integer — the descriptor it is parked on. Pay that tax ten thousand times and the machine drowns in bookkeeping for connections that are, almost all the time, doing nothing. So here is the plan. We take the single fact a waiter needs to remember and store it in a small object on the heap instead of a kernel stack. That object is a coroutine, which is nothing more exotic than an ordinary stack frame that has been refused permission to die. await is the point where such a frame pauses and hands control back. The event loop is one thread running a ready-queue, resuming whichever frame's awaited thing has just become ready. All the way through, we keep asking the one thing that actually matters — how does a single thread wait on ten thousand sockets at once without any one of them blocking the rest? By the end we have traded 78 gigabytes of idle stacks for a few megabytes of parked frames. One epoll call watches over them, sleeping on every socket at once and waking only for the ones that are ready. No second thread ever runs. The machine simply remembers exactly where each waiter stopped, and picks it back up the instant its byte lands.

01The need: a stack per waiter is wasteful

Let's start with a job concrete enough to feel. You're building a chat server, and the requirement is simple to state: hold ten thousand connections open at once. Not ten thousand messages a second — ten thousand sockets. Each one is a person who has connected and is now mostly just… there. Typing a little, reading a little, idle for whole seconds between keystrokes. Every one of those connections is a file descriptor — a small integer the kernel hands you to name one open socket — waiting for its next byte. Your only job is to be ready the instant that byte lands.

Reach for the tool the last chapter handed you and the answer looks obvious: one thread per connection. Each thread calls os.read(fd) and the read blocks. The thread now sits there, parked by the kernel, doing nothing until a byte shows up to wake it. It's clean and linear, and every connection reads like its own little synchronous story. Now turn up the dial. You test it with fifty clients and it purrs. You push it to ten thousand and the machine falls over: memory balloons, the CPU pins itself doing nothing you asked for, and latency turns to porridge. The model didn't just slow down, it collapsed. To replace it, you first have to know exactly what one blocked thread costs, counted in bytes and in microseconds.

Start with memory. Every OS thread owns a full call stack. On Linux the default reservation is 8 MiB of virtual address space per pthread, and you can read it with ulimit -s. Only the pages you actually touch get backed by physical RAM, so a thread with a shallow call chain might genuinely use just tens of kilobytes. Even so, ten thousand of them is hundreds of megabytes spent on threads whose entire occupation is holding still. And the reserved 8 MiB still costs you address space and page-table entries. Do the arithmetic and it stops being abstract.

# what 10,000 thread stacks *reserve*, before a single byte is read
waiters      = 10_000
stack_mib    = 8                 # Linux default: ulimit -s -> 8192 KiB
reserved_gib = waiters * stack_mib / 1024
print(f"reserved address space: {reserved_gib:.0f} GiB")
# reserved address space: 78 GiB

# the same 10,000 waiters as suspended heap objects instead
coro_bytes = 416                 # a parked coroutine + its frame, order-of-magnitude
heap_mib   = waiters * coro_bytes / 1024 / 1024
print(f"heap for coroutines:    {heap_mib:.1f} MiB")
# heap for coroutines:    4.0 MiB

Now the CPU. Every thread is an entry in the kernel's run queue, and the scheduler must consider each one. Every time it switches from thread A to thread B it performs a context switch: save A's registers, load B's, swap the stack pointer, and — the expensive part — poison the caches the CPU had warmed up. The new thread's code and data aren't in L1 or L2, and its address translations aren't in the TLB, so it stalls fetching them back from slower memory. A single switch is on the order of one to two microseconds of pure overhead. With thousands of threads waking on scattered I/O, the machine spends more of its day switching than working. This is the classic thundering herd: a burst of readiness wakes hundreds of threads that all fight for the same few cores.

And there is a third cost you already met: the GIL. CPython threads cannot run Python bytecode in parallel — only one holds the interpreter lock at a time. So a thread-per-connection design buys you concurrency of waiting, never concurrency of computing. You pay the full price of OS threads and get only the cheap half of what threads are for.

Here is the observation that cracks the whole problem open. A thread blocked in read() is holding an entire call stack and a scheduler slot just to remember one fact: "I am waiting on file descriptor 42." That is a huge amount of machinery to represent one integer of waiting. The fix writes itself the moment you say it out loud. Don't give each waiter a kernel stack. Give it a small object on the heap holding just that one fact, and use one thread to watch all the descriptors at once. Thousands of cheap, mostly-idle waiters is the pressure, and every design decision in the rest of this chapter is forced by it.

So the cheap waiter has to be an object on the heap that remembers "where I was." You already know what remembers where a function was — a stack frame. The next section is the whole trick: a coroutine is that frame, kept alive instead of destroyed. →

02A coroutine is a parked frame

The last section demanded a cheap waiter: a small object on the heap that remembers one thing — where I was, and what I was holding. Now make it concrete. What object is that? You already know the answer from the functions chapter; you just haven't been told it can outlive its call. When you call a normal function, Python pushes a stack frame. That frame holds the call's local variables, its temporary value stack, and f_lasti, the offset of the last bytecode instruction it ran — precisely "where I am." On return, that frame is popped and freed, and the locals vanish. A coroutine is that identical frame, refused permission to die.

Here is the pivot, and it is smaller than it sounds. An async def does not compile to a normal function. The compiler sets a flag — CO_COROUTINE — on the code object, and that flag changes what calling it does. Call an ordinary function and its body runs. Call a coroutine function and zero of its body runs. Python allocates a PyCoroObject on the heap and hands it straight back to you. The code you wrote hasn't executed at all yet. Prove it to yourself:

async def greet(name):
    print("running now")      # does NOT print on the call below
    reply = await ask(name)
    return reply

c = greet("Ada")   # body has NOT run — no output yet
print(c)             # <coroutine object greet at 0x7f2a...>
print(type(c).__name__)  # coroutine
print(c.cr_frame.f_lasti)  # -1  (not started; f_lasti is the resume offset)

You called greet("Ada") and got back not a greeting but an object — a paused function you can hold in a variable, pass around, or drop. Inside it sits its frame (cr_frame), and that frame owns the locals array, the value stack, and f_lasti. It also carries a state, the coroutine's whole life story in four words: CREATED when freshly built, SUSPENDED at each await, EXECUTING while its bytecode is actually running, and CLOSED once it has returned or been finalized. Because the frame is now a heap object, it is reference-counted like everything else. As long as your variable c points at the coroutine, the frame — every local, the value stack, the resume offset — stays alive. When the refcount hits zero, the frame is finalized. The frame lives exactly as long as the object does, no longer.

How do you make it run, then resume it? With the low-level driver underneath everything else in this chapter: send. A fresh coroutine is started with c.send(None), which resumes the frame at f_lasti, runs bytecode until the next suspension point, and returns control to you. Call send again and it picks up from exactly that offset, locals intact, mid-flight. When the coroutine finally returns, its return value is delivered to you wrapped in a StopIteration:

async def add_later():
    x = 3                 # a local that must survive the pause
    y = await pending      # suspends here; frame is parked on the heap
    return x + y

c = add_later()
c.send(None)            # runs x=3, hits await, SUSPENDS — control returns to us
print(c.cr_frame.f_locals)  # {'x': 3}  — x is frozen in place, mid-function
try:
    c.send(40)         # resumes: y becomes 40, runs return 43
except StopIteration as stop:
    print(stop.value)  # 43  — the return value rides out on StopIteration

Sit with the middle line. After the first send, the function is stopped inside its own body — x equal to 3, sitting in a frame on the heap, with no thread of any kind holding it there. A blocked thread parks that same kind of state, a few locals and a resume point, inside an 8 MiB kernel stack and a scheduler slot. A suspended coroutine parks the identical state in a 400-byte object. That is the entire cost collapse from section 1, made real: suspension without a thread. No kernel involvement, no run-queue entry, just a Python object on the heap that happens to remember where it stopped.

Driving a frame by hand with send is not a program. And notice we glossed the crucial line: what did await pending actually do, and what value came back out when the frame suspended? That returned value is how a coroutine tells the world what it is waiting for. →

03await is a chosen yield point

You can pause a frame by hand now, but two questions are still open, and they are the two that make async a system rather than a party trick. First: what does await actually compile to? Second, and this is the one that matters: what value comes back out when a coroutine suspends? That value is the coroutine's entire vocabulary for talking to the outside world. It is how a parked frame says, "here is the specific thing I am waiting for; resume me when it's ready." Get this, and the event loop in the next section becomes obvious.

Start with the mechanism. await x requires x to be awaitable, which means type(x) defines __await__. Under the hood, await calls x.__await__() to get an iterator and then drives that iterator. Here is the load-bearing fact: whatever that iterator yields, await propagates straight up and out of the current coroutine. Coroutines and generators share one suspend/resume engine. A suspension is literally a yield that travels up through every awaiting frame until it exits the top coroutine as the return value of send. So "what does the driver see when a coroutine awaits?" has a precise answer: it sees the object that reached the bottom of the await chain and yielded.

In asyncio, that object is a Future — a placeholder for a result that isn't ready yet. A leaf await, like waiting on a socket, ultimately runs yield self from a Future's __await__, guarded by "only if I'm not already done." That Future object surfaces out of the coroutine as a message to the driver: "park me, and resume me when this Future is resolved." You can build the smallest possible version by hand and watch the token travel:

class WaitFor:
    def __init__(self, token):
        self.token = token
    def __await__(self):
        value = yield self.token   # THIS is the suspend; token exits the coroutine
        return value              # the resume value becomes the value of `await`

async def job():
    got = await WaitFor("need: fd42")
    return f"resumed with {got}"

c = job()
out = c.send(None)     # out == 'need: fd42'  — the token surfaced to us, the driver
try:
    c.send("ok")          # 'ok' threads back down to become `got`
except StopIteration as s:
    print(s.value)  # resumed with ok

The token you yielded came out of send, and the value you sent back in became the result of the await. That round trip — object out, result in — is the whole protocol the event loop speaks. Now a subtlety that trips up nearly everyone: awaiting another coroutine does not touch the loop. When you write await handler(), you are not yielding to any driver. You are splicing handler's frames onto the same suspend chain, and it becomes one taller stack of paused frames. Only when execution reaches a leaf — a Future that runs yield self — does anything actually surface to the loop. Coroutines awaiting coroutines build the tower; a Future at the bottom is the only thing that yields off the top of it.

And here is the conceptual payoff, the reason this design feels different from threads. Control only ever leaves a coroutine at an await. Between two awaits, the coroutine runs to completion with nothing able to interrupt it — no timer, no preemption, no other coroutine sneaking in. The coroutine names its own pause points. That is the deal async makes with you. In exchange for choosing where you yield, you get to treat every stretch of code between two awaits as atomic. The torn reads and lost updates that haunted the threading chapter simply cannot happen there, because no second coroutine can observe your half-finished state. The bill for that gift comes due in the last section: you are now responsible for awaiting often enough.

A coroutine suspends and yields a Future; you've been resuming it by hand. That doesn't scale to ten thousand. Next: the thing that does the resuming for you — and it is nothing more than a while-loop with a queue. →

04The event loop is one thread and a ready-queue

You now have a coroutine that suspends and hands you a Future, and you have been the driver, calling send by hand. That does not survive contact with ten thousand connections. You need the thing that drives automatically, and you need it demystified, because "the event loop" sounds like a runtime cathedral you could never build. It isn't. It is a while loop over a queue, and by the end of this section you could write it yourself. Once you see that, asyncio stops being a black box and concurrency-without-threads becomes plainly mechanical.

At its core, asyncio's loop is this: while running, run every callback currently in a FIFO ready-queue. The queue — loop._ready — is a collections.deque. Each entry is a Handle, a plain "call this function with these args" record produced by loop.call_soon(fn, *args). That deque, one thread, and one loop is the entire engine, and nothing in it understands coroutines. But a coroutine isn't a callback, so what bridges the two? The Task.

asyncio.create_task(coro) wraps a coroutine in a Task and immediately schedules the Task's __step method with call_soon. When __step runs, it calls coro.send(None), driving the coroutine forward until it yields a Future. Then the Task makes the move that everything hinges on. It does not spin, poll, or re-queue itself. It registers a callback on that Future — future.add_done_callback(self.__step) — and returns. Now the Task is off the ready-queue entirely. It is asleep, remembered only by the Future it is waiting on. The whole thing is short enough to hold in your head:

from collections import deque

class Loop:
    def __init__(self):        self._ready = deque()
    def call_soon(self, fn, *a): self._ready.append((fn, a))
    def run(self):
        while self._ready:               # the entire engine, right here
            fn, a = self._ready.popleft()
            fn(*a)

class Future:
    def __init__(self, loop):
        self.loop, self.done, self._cbs = loop, False, []
    def add_done_callback(self, cb):
        self._cbs.append(cb) if not self.done else self.loop.call_soon(cb, self)
    def set_result(self, value):
        self.result, self.done = value, True
        for cb in self._cbs:            # resolving wakes the parked Task
            self.loop.call_soon(cb, self)

class Task:
    def __init__(self, coro, loop):
        self.coro, self.loop = coro, loop
        loop.call_soon(self.step)      # schedule the first step
    def step(self, *_):
        try:
            fut = self.coro.send(None)  # run to the next await -> a Future
        except StopIteration:
            return                    # coroutine finished; Task is done
        fut.add_done_callback(self.step) # park on it; re-queue when it resolves

Trace one Task's life through that code and you get a clean cycle. It starts on the ready-queue. The loop pops it and runs step, which drives the coroutine forward until it yields a Future. The Task registers step as that Future's done-callback and vanishes. Time passes and other Tasks run. Eventually something calls set_result on the Future, which walks its done-callbacks and call_soons each one, putting step back on the ready-queue. On the next turn the loop pops it, step calls send again, and the coroutine resumes exactly where it suspended. Ready-queue → run to await → parked on a Future → Future resolves → back on ready-queue. Every coroutine in the system rides that loop.

Now concurrency becomes precise, not hand-wavy. With a thousand Tasks alive, at any instant the ready-queue holds only the few whose awaited thing just became ready. The loop runs each of those forward to its next await, then moves on to the next one. Progress interleaves on one thread, in slices, each slice ending at a chosen yield point. Say the words carefully, because the distinction is the soul of the chapter. This is concurrency — many operations in flight, advanced a slice at a time — and emphatically not parallelism. One core, one thread, exactly one coroutine's bytecode executing at any moment. Ten thousand things in progress; one thing running.

One hole remains. If every Task is parked on a Future waiting for a socket, the ready-queue empties and the loop has nothing to run. Who resolves an I/O Future? How does one thread learn that three of ten thousand sockets just went readable — without blocking on any of them? →

05One syscall watches all the sockets

The loop from the last section has a hole you can feel. Suppose every Task is parked on a Future, each waiting for a socket to deliver bytes, and the ready-queue is empty. The while self._ready loop would find nothing to run and simply exit. Or, if you wrote it to keep going, it would spin the CPU at 100% checking an empty deque forever. Neither is a server. Something has to resolve those I/O Futures, and it has to do so for one thread watching ten thousand sockets, without blocking on any single one. This is the exact syscall that pays off section 1's promise: one thread watches all the descriptors.

Recall the file-descriptor model. An open socket is just an integer index into a kernel table. A blocking read(fd) parks the calling thread until fd has data — one thread, one fd, which is precisely the model that collapsed. The kernel offers a far better primitive: readiness multiplexing. Instead of "block this thread on this one fd," you hand the kernel a whole set of descriptors and ask a different question: "which of these are ready right now?" Three generations of this exist: select() (portable, but O(n) in the number of fds and capped at a small limit), poll() (no cap, still O(n)), and Linux's epoll (O(number ready), scaling to hundreds of thousands of connections). Python wraps all of them behind the selectors module, whose DefaultSelector picks the best one available — epoll on Linux, kqueue on BSD and macOS, select on Windows.

Here is how that completes the loop turn. When a coroutine awaits a socket read, asyncio does not block. It calls loop.add_reader(fd, callback), which registers the fd with the selector (selector.register(fd, EVENT_READ, ...)) and leaves the Task parked on its Future. So the loop's turn is now precisely two steps: run the ready-queue until it is empty, then call selector.select(timeout) — a single syscall that puts the one thread to sleep on all registered descriptors at once. When any of them becomes readable, the kernel wakes the thread and hands back the list of ready fds. For each one, the loop fires the registered callback, which resolves that fd's Future via set_result, which — you know the rest — call_soons the parked Task back onto the ready-queue. You can watch the raw primitive without any of asyncio in the way:

import selectors, socket

sel = selectors.DefaultSelector()   # epoll on Linux, kqueue on macOS

def watch(sock):
    sock.setblocking(False)
    sel.register(sock, selectors.EVENT_READ, data=sock)

while True:
    events = sel.select(timeout=2.0)  # ONE syscall sleeps on every fd at once
    if not events:
        print("woke on timeout, nothing ready")  # ~0% CPU while it slept
        continue
    for key, mask in events:      # only the READY ones come back
        data = key.data.recv(4096)
        print(f"fd {key.fd} readable: {len(data)} bytes")

That timeout argument is not a guess. asyncio keeps every scheduled timer and sleep in a heapq of TimerHandles, ordered by wake time. Before each select, it computes the delay until the nearest timer and passes that as the timeout. So await asyncio.sleep(2) is implemented as "put a timer 2 seconds out on the heap, and make sure select wakes the thread by then." I/O readiness and timed sleeps resolve through the same one call. select returns either because a socket went ready or because the next timer came due, whichever happens first.

Stand back and see the payoff whole. One thread. One epoll_wait. Ten thousand sockets. When nothing is ready, the thread sleeps inside the kernel at zero CPU — not spinning, genuinely blocked, costing you nothing. When descriptors become readable, the thread wakes and resumes exactly and only the coroutines whose fd just went ready. The cost of a turn scales with the number of ready events, not the number of connections. That is the entire reason epoll exists, and the entire reason ten thousand idle clients are nearly free. The N kernel stacks of section 1 have collapsed into a single selector call.

The model is complete and it is beautiful — which is exactly the trap. The single most common async bug is a coroutine that looks fine and quietly freezes the entire server. Time to meet the sharp edge of the cooperative deal. →

06The sharp edge: one blocking call freezes everything

You trust the model now, and that trust is the trap. The most common async bug in existence is a coroutine that looks completely fine — it calls time.sleep, or requests.get, or runs a heavy computation — and mysteriously wedges the whole server. Heartbeats stop, other connections hang, throughput drops to nothing, and there is no exception to point at. This is not a bug in asyncio. It is the unavoidable flip side of the cooperative deal from section 3. Once you can see why, you can spot it in a diff and fix it in one line. This section is the difference between async code that scales and async code that hangs.

Reassemble the whole machine in one breath. There is one thread. The loop advances a coroutine only by calling send, and send does not return until the coroutine reaches its next await. Crucially — and this is the entire point — there is no preemption. The OS scheduler yanks control away from a thread every few milliseconds whether it cooperates or not (the threading chapter's timer interrupt). Nothing does that to a coroutine: nothing will ever interrupt a running one mid-function. So the loop is blocked inside send for exactly as long as the running coroutine takes to reach its next await. Every failure mode falls straight out of that one sentence.

time.sleep(5) is a blocking syscall that parks the one and only thread in the kernel for five seconds. Not "this coroutine" — the thread. During those five seconds the loop cannot pop the ready-queue, cannot call select, cannot resolve a single Future. Ten thousand sockets could go readable and the thread wouldn't notice, because the thread that would notice is asleep. A CPU-bound loop — hashing a file, summing a huge range, parsing a big document — is the same disease without a syscall. It never executes an await, so send simply does not return for the whole computation, and the ready-queue and selector sit untouched the entire time. And requests.get is the subtle killer. It does a blocking socket read on the loop thread instead of yielding a Future, so it stalls everything for the full network round-trip — the precise thing async was built to avoid, smuggled back in by an innocent-looking import. See it and un-see it side by side:

import asyncio, time

async def heartbeat():
    while True:
        print("beat")
        await asyncio.sleep(0.5)   # yields a timer Future — loop keeps turning

async def bad():
    time.sleep(3)              # parks the ONE thread — every beat freezes 3s

async def good():
    await asyncio.sleep(3)      # parks THIS coroutine — beats keep going

async def cpu_bound():
    total = await asyncio.to_thread(sum, range(10**8))  # push the grind off the loop
    return total

The rule crystallizes to four words: async code must await, never block. The fixes are exact, not vibes. First, replace blocking library calls with their awaiting equivalents — await asyncio.sleep instead of time.sleep, an async HTTP client like aiohttp or httpx instead of requests. Now the wait becomes a Future the loop can park on, and the thread stays free to run everyone else. Second, when the work is genuinely blocking or CPU-bound and cannot be made awaitable, push it off the loop thread with await loop.run_in_executor(...) or its friendly wrapper asyncio.to_thread(...). Those hand the work to a thread- or process-pool and give you back a Future. The pool thread blocks or grinds, and the loop thread keeps turning. (This is also the one place threads and async cooperate rather than compete — and where the GIL still matters: use a process pool, not a thread pool, for CPU-bound work, or the GIL will serialize it right back.)

asyncio even ships a canary for exactly this. Run your program with asyncio.run(main(), debug=True) and the loop times every callback. Any that runs longer than 100 ms gets logged — Executing <Handle ...> took 0.512 seconds. That turns the invisible freeze into a visible warning pointing at the offending coroutine. Turn it on the first time a server feels mysteriously sluggish, and the culprit usually names itself.

End on the symmetry, because it is the honest summary of the whole chapter. Preemption — threads — costs you races: any thread can be interrupted mid-update, so shared state needs locks. But it rescues you from any single stuck task, because the scheduler always takes control back. Cooperation — async — gives you race-free code between awaits, because nothing interrupts you. But it makes every coroutine personally responsible for yielding, because nothing takes control back if you don't. Same coin, two faces. You traded the OS's rescue for the OS's races, and the price of that trade is a single discipline: await often, block never.