26Threads & the GIL — derived from refcounting

You have heard both halves of the folk wisdom, usually in the same breath: threads make Python faster, and threads don't help in Python. Both are true. The gap between them is where a beginner loses a weekend and blames the language. Back in Chapter 3 we gave every object a header carrying an ob_refcnt counter that ticks up and down as names come and go. It looked like private bookkeeping. In this chapter we take that one counter seriously across two cores, and it turns out to decide whether your program can use a second core at all. Here is the plan. We point two threads at the same heap, watch a refcount race quietly corrupt memory, and see CPython's answer — the Global Interpreter Lock — fall out of the wreckage. It is not a wart someone forgot to delete; it is a theorem. The whole way through, we keep asking the one question that settles the folklore: given refcounting, could Python ever safely let two threads run bytecode at once? By the end we can look at any slow program and say, in a single measurement, whether threads will rescue it or do nothing at all. The same lock that makes threads useless for computing leaves them untouched for waiting, and we will know exactly why.

01The one-question diagnosis

Let's start with the trap, because almost everyone falls into it once. You read that threads make programs faster, so you wrap a loop in a ThreadPoolExecutor and run it. It comes back slower. Not a little slower — measurably, embarrassingly slower, fans spinning up for nothing. The instinct is to decide Python is broken, or that you held it wrong. Neither is true. You reached for the right tool on the wrong workload. The difference between the two workloads isn't a matter of taste; it's physical. It lives in what the CPU is actually doing while the wall clock ticks. So before we write a single thread, we need the one question that sorts any program cleanly into one bucket or the other. Every later decision in this chapter falls straight out of the answer.

Here's the fork. A running program spends its wall-clock time in exactly one of two states, and down at the silicon they could not be more different. In the first, the core is busy computing. The instruction pointer never leaves your code. The pipeline retires an arithmetic, logic, or branch instruction essentially every cycle. The ALU is hot. Summing a hundred million integers, resizing an image, hashing a password — this is CPU-bound work, and the only way to finish sooner is more cycles per second, or more cores working the problem at the same time. In the second state, the core is waiting. Your thread issued a system call — read(), recv(), connect() — and the kernel put it to sleep in a wait queue (Linux calls this the uninterruptible-sleep or D-state). The CPU is not slow here. It is idle for this task, descheduled, doing other people's work or nothing at all. It will only wake your thread when an interrupt fires: a packet arrives, a disk DMA transfer completes. Downloading fifty pages, querying a database, reading files — this is I/O-bound work, and the bottleneck is not computation at all. It is the waiting.

So the diagnostic question — call it the crunch test — is this: if the network and the disk were infinitely fast, would this program finish instantly? If yes, the time was spent waiting. The work is I/O-bound, and the win is to overlap the waits: have many requests in flight so their idle stretches stack on top of each other instead of running end to end. If instead the only cure is a faster CPU, the work is CPU-bound. No amount of overlapping helps, and only true parallelism — more cores actually executing — moves the needle. That single fork decides everything: threads, async, or processes.

You do not have to guess which state you are in. You can measure it, because the operating system already keeps two different clocks. time.perf_counter() measures wall time — elapsed reality, including every second spent asleep in a kernel queue. time.process_time() measures only CPU time actually charged to your process — the cycles you truly burned. Hold them up against the same function and the ratio tells you the workload:

import time

def fetch_url(url):            # pretend network: the CPU does almost nothing
    time.sleep(2)              # blocked in the kernel — not computing
    return f"<html>{url}</html>"

def crunch(n):                 # pure arithmetic: the ALU never rests
    return sum(i * i for i in range(n))

def classify(fn, *args):
    wall0, cpu0 = time.perf_counter(), time.process_time()
    fn(*args)
    wall = time.perf_counter() - wall0
    cpu  = time.process_time()  - cpu0
    verdict = "CPU-bound" if cpu / wall > 0.5 else "I/O-bound"
    print(f"{fn.__name__:9} wall={wall:5.2f}s cpu={cpu:5.2f}s  {verdict}")

classify(fetch_url, "example.com")   # wall=2.00s cpu=0.00s  I/O-bound
classify(crunch, 10_000_000)         # wall=0.80s cpu=0.80s  CPU-bound

Read the two comment lines like an X-ray. The fetch burned two full seconds of wall time and essentially zero CPU — the process was asleep, the core free. The crunch spent its wall time and its CPU time in lockstep: every second of clock was a second of computation. That cpu / wall ratio is the whole diagnosis in one number. Near 1 means you were computing; near 0 means you were waiting. Everything the rest of this chapter says about the GIL is downstream of this split, because — as you will see — the GIL punishes the crunch and leaves the waiting untouched.

The classifier says overlap the waits for I/O and use more cores for compute. But to see why one works in Python and the other refuses to, you first have to know what a thread actually is — and what, alarmingly, it shares. →

02A thread is a flow sharing the heap

To predict what threads do, you have to know precisely what a thread is — and, more sharply, what it is not. The dangerous half is what a thread does not get: its own copy of your data. In Volume 1 every object lived on the heap behind a header — ob_refcnt, ob_type, then the payload — at a real address in RAM, and every name was a pointer to it. When you spin up a second thread, none of that is duplicated. The list is not cloned. The dict is not cloned. Your module globals are not cloned. Both threads reach into the same heap and touch the same objects.

Formally, a process is an address space plus one or more threads of execution. Each thread is scheduled independently by the OS kernel, and each thread owns exactly three private things: its own call stack (its chain of frame objects, its local variables), its own instruction pointer (which line it is on), and its own register set, the handful of CPU registers the kernel saves and restores every time it switches threads. That is the entire list of what is private. Everything else is common property: the heap and every Python object on it, the loaded bytecode, the open file descriptors, the interpreter's own state. So when thread A and thread B both name results, they are not holding two similar lists. They hold the identical 8-byte pointer to the identical header at the identical address. One list, two arrows.

Now add the part that turns “shared” into “dangerous.” The kernel does not switch threads politely at the end of your Python statements. It switches on a timer interrupt, at an arbitrary machine instruction — possibly halfway through evaluating a single expression — saving A's registers and restoring B's. This interleaving is invisible from your source code, and it is non-deterministic run to run. A thread can be frozen mid-reach, its half-finished work exposed, while another thread walks up to the very same object and starts changing it. Hold that image: two arrows into one list on one heap, and a scheduler that can pause either arrow between any two instructions. Every race condition in the chapter is already implied by that picture.

You can prove the sharing is literal — not a metaphor — in five lines, using id(), which in CPython returns the object's actual heap address, and threading.get_ident(), which returns the OS-level identity of the running thread:

import threading

shared   = []            # one list object, at one address on the heap
seen_ids = []

def worker():
    shared.append(threading.get_ident())   # which thread am I?
    seen_ids.append(id(shared))            # what address is the list at?

t1 = threading.Thread(target=worker)
t2 = threading.Thread(target=worker)
t1.start(); t2.start()
t1.join();  t2.join()

print(shared)      # [140192..., 140188...]  — two DIFFERENT thread ids
print(seen_ids)    # [4501...840, 4501...840] — ONE identical address

Two different thread identities, appended into one list, which both threads saw at the same address. That is the whole point compressed into two numbers: the flows of control are separate, but the data underneath them is one and the same. Notice too that neither thread had to be handed the list. shared was simply a global, and globals are just names in the module object, which lives on the shared heap. There is no per-thread copy of your module. Everyone reads and writes the one original.

This is exactly where the metal-up thread of Volume 2 splits in two. When you cross a process boundary — the subject of chapter 28 — fork() copies the entire address space. So processes share nothing by default and can never corrupt each other's objects. What passes between them is bytes, never live objects, which is precisely why processes escape the GIL. Threads are the opposite border: nothing is copied, everything is shared, and live objects are touched directly by two cores at once. That single structural difference — copy-everything versus share-everything — is the root of every behavior in this chapter, including the lock we are about to prove must exist.

Two threads, one heap, a scheduler that pauses either one between any two instructions. Now aim both of them at the one field every object carries — ob_refcnt — and watch the crash assemble itself. →

03Why the GIL must exist

This is the spine of the chapter, and its payoff. You have been told the GIL is an embarrassment — a bottleneck someone should simply delete. By the end of this section you will see it is nothing of the kind. It falls straight out of Volume 1's refcounting as an inevitability. Take refcounting seriously across the two shared-heap threads you just met, and a global lock — or a lock on every single object — stops being a design choice. It becomes forced.

Recall the mechanism from chapter 3. Every object carries ob_refcnt, a count of how many references point at it. Every time a reference is created or destroyed — an assignment, passing an argument, returning a value, a name going out of scope — CPython runs Py_INCREF or Py_DECREF. And here is the fact that detonates everything: at the machine level, incrementing a counter in memory is not one instruction. It is three. LOAD the current count from RAM into a CPU register, ADD one to the register, then STORE the register back to RAM. A read, a modify, a write. And as you learned last section, the scheduler can fire a timer interrupt between any two of them.

Now interleave two threads over one object whose count is 2. Thread A executes LOAD and pulls 2 into its register. The timer fires. The kernel freezes A — its register still privately holding 2 — and runs Thread B. B does the whole tick cleanly: LOAD 2, ADD to 3, STORE 3. The count in RAM is now correctly 3. The kernel switches back to A, which has no idea anything happened. It resumes from where it stood, with its stale 2, ADDs to get 3, and STOREs 3. Two increments occurred, so the count should read 4. It reads 3. One increment vanished into the gap — a classic lost update.

A lost increment is bad. The mirror case is catastrophic. Run the same race on Py_DECREF and you can lose a decrement the same way. Worse, an interleaving can drive the count to 0 while a live reference still exists. Reaching 0 is CPython's signal to free the object: call its destructor, return its memory to the allocator. So the object is torn down and its bytes recycled while another thread is still holding a pointer into it. The next time that thread dereferences its now-dangling pointer, it reads freed memory — a use-after-free. That is not a wrong answer; that is a segfault, or worse, silent corruption. The very mechanism that makes CPython's memory management deterministic becomes, under naive threading, a machine for crashing.

You can watch the lost-update half happen in pure Python, no C required, by spelling the read-modify-write out by hand so the interpreter can be preempted between the pieces:

import threading

counter = 0

def bump():
    global counter
    for _ in range(1_000_000):
        tmp = counter        # LOAD  — read the count into a local
        tmp = tmp + 1        # ADD   — bump the private copy
        counter = tmp        # STORE — write it back; preemptible before this

a = threading.Thread(target=bump)
b = threading.Thread(target=bump)
a.start(); b.start(); a.join(); b.join()

print(counter)   # NOT 2_000_000 — something like 1_370_544. Updates were lost.

Two million increments go in; well under two million come out. Every shortfall is one thread's LOAD stomped by the other's STORE — the exact race, made visible in your own code. And ob_refcnt is bumped this way not once per million iterations but on every reference operation the interpreter performs — millions of times a second, on every object alive. You can even watch the count move:

>>> import sys
>>> x = object()
>>> sys.getrefcount(x)     # 2: your name x, plus getrefcount's own argument
2
>>> y = x                    # a second name — the count ticks up
>>> sys.getrefcount(x)
3
>>> del y                     # drop it — the count ticks down
>>> sys.getrefcount(x)
2

So the fix space has exactly two shapes. Option one: make every refcount tick atomic, or give every object its own lock. Correct — and ruinous. There are millions of tiny objects, so a per-object lock on each is enormous overhead, and atomic increments force the CPU to serialize the cache line on every tick, which murders performance on the single-threaded programs that are the overwhelming majority. Option two: a single Global Interpreter Lock that a thread must hold to touch any Python object. Then refcount ticks are never concurrent, because only one thread runs at a time. CPython chose option two. One mutex. Near-zero cost in the common single-threaded case, and refcounting — hence deallocation, hence all of C-level memory safety — stays sane. The GIL is the price of refcounting without per-object locks.

The lock must exist — accepted. But “the GIL” is still a fog until you know the exact rule for who holds it and when they hand it off. That rule explains every surprising thing threads do next. →

04One lock, one runner at a time

You have accepted the lock. Now you need its operating manual, because without the precise handoff rule “the GIL” stays a vague fog and threads keep surprising you. The confusion to kill first: are Python threads even real? Yes. Emphatically. threading.Thread creates a genuine operating-system thread — a pthread on Linux and macOS, a native thread on Windows. The kernel schedules them, they can be placed on different physical cores, and they are not a simulation. What they cannot do is run the interpreter simultaneously, because advancing the bytecode evaluation loop requires holding the GIL, and there is one GIL. Real threads, one baton; they take turns carrying it.

The rule in a single line: a thread must hold the GIL to execute Python bytecode. Everything else is the mechanics of how the baton gets passed, and it passes in exactly two ways.

The first is voluntary release. Whenever a thread is about to make a blocking call into C — a syscall, time.sleep, a socket recv, a file read — CPython wraps the call in the macro pair Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS. The first macro drops the GIL right before the thread goes to sleep in the kernel; the second reacquires it when the call returns. The logic is obvious once you see it. A thread that is about to block for two seconds has no use for the interpreter lock during those two seconds, so it lets go, and some other thread gets to run. This one mechanism is the entire reason I/O-bound threading works — hold that thought for the next section.

The second is involuntary preemption. A thread doing pure arithmetic never makes a blocking call, so it would never voluntarily let go — it could hog the baton forever and starve every other thread. To prevent that, the interpreter forces a checkpoint. Since Antoine Pitrou's 3.2 redesign, CPython uses a time-based switch interval, default 5 milliseconds, which you can read with sys.getswitchinterval(). A background timer sets a flag — the eval_breaker — when the interval elapses. The running thread checks that flag at the boundary between bytecode instructions. When it sees the flag set, it releases the GIL and signals a waiting thread to take it. And here is the load-bearing detail: the release happens only between bytecode instructions, never in the middle of one.

>>> import sys
>>> sys.getswitchinterval()      # seconds a thread may hold the GIL before a forced checkpoint
0.005
>>> import dis
>>> dis.dis("x += 1")          # one Python line — several bytecodes
  LOAD_NAME     0 (x)
  LOAD_CONST    0 (1)
  BINARY_OP    13 (+=)
  STORE_NAME    0 (x)

Now the two halves click together. Because the GIL is released only between bytecodes, any operation that is a single bytecode is effectively atomic — no other thread can wedge itself into the middle of it. A dict[key] = value insert compiles to one STORE_SUBSCR. It cannot be interrupted mid-store, so concurrent inserts into one dict never corrupt the dict's internal structure. But x += 1, as the disassembly shows, is four bytecodes: load, load, add, store. The scheduler can preempt between the ADD and the STORE, and you are back in the exact lost-update race from section 3 — on your own variable this time. This is the single most misunderstood thing about the GIL. It protects ob_refcnt, the interpreter's private bookkeeping, so the interpreter never crashes. It does not protect the logical invariants of your program. Your counter, your list-of-results, your two-line “read balance, then write balance” — those span multiple bytecodes and remain fully racy.

One more confirmation that these are real kernel threads and not green threads: they share a process id, but each has its own OS-level identity. And two of them pinned to pure computation still deliver only one core's worth of throughput, because the baton, not the cores, is the bottleneck.

import os, threading

def who():
    print(f"pid={os.getpid()}  native_id={threading.get_native_id()}")

who()
t = threading.Thread(target=who); t.start(); t.join()
# same pid (one process), different native_id (two real OS threads)

You now hold the two release rules — voluntary on blocking, forced every 5ms. Point them at the two workloads from section 1 and the folklore finally resolves into arithmetic: why compute threads flatline and wait threads fly. →

05The crunch test: I/O overlaps, CPU doesn't

Theory earns its keep only when it predicts a measurement you can run yourself. So take the two release rules from the last section — voluntary drop on blocking, forced 5ms handoff otherwise — and push both workloads from section 1 through them. This is where “I added threads and it got slower” finally gets its full explanation, and where you get to watch the two outcomes diverge.

Start with CPU-bound. Two threads each sum a hundred million integers. Neither ever makes a blocking call, so neither ever voluntarily releases the GIL. The only handoff is the forced 5ms checkpoint, and while one thread holds the baton, the other is parked, not computing. So the total arithmetic is unchanged, and now it is serialized through one lock. Two threads do not split the work; they take turns doing all of it. Wall time comes out roughly equal to the single-threaded time, plus the overhead of thousands of GIL acquire/release handoffs and context switches. That is why it often runs slightly slower, and never near half. Run it on 1, 2, 4 threads and the wall-time line stays flat, or drifts upward. That flat line is the definitive fingerprint of the GIL.

import time
from concurrent.futures import ThreadPoolExecutor

def crunch(n):
    return sum(i * i for i in range(n))

N = 30_000_000

t = time.perf_counter()                       # sequential: do it twice, back to back
crunch(N); crunch(N)
print("2x sequential:", round(time.perf_counter() - t, 2), "s")

t = time.perf_counter()                       # two threads: hoping for half the time
with ThreadPoolExecutor(max_workers=2) as pool:
    list(pool.map(crunch, [N, N]))
print("2 threads    :", round(time.perf_counter() - t, 2), "s")
# 2x sequential: 4.10 s      2 threads: 4.30 s   — no speedup, a little worse

Now I/O-bound, same harness shape, but the work is a blocking wait instead of arithmetic. Each thread's time.sleep(2) — physically identical to a socket wait — hits Py_BEGIN_ALLOW_THREADS and drops the GIL before parking in the kernel. So while thread A sleeps, the baton is free. Thread B grabs it and starts its own sleep, dropping the baton again for C, and so on. All the waits are in flight at once. Four two-second sleeps that took eight seconds end to end now finish in about two, because the idle time — which was the entire cost — overlaps for free.

import time
from concurrent.futures import ThreadPoolExecutor

def fetch(url):
    time.sleep(2)          # blocking wait — GIL released the whole time
    return url

urls = ["a", "b", "c", "d"]

t = time.perf_counter()
for u in urls: fetch(u)              # sequential: waits stack end to end
print("sequential:", round(time.perf_counter() - t, 2), "s")   # ~8.0 s

t = time.perf_counter()
with ThreadPoolExecutor(max_workers=4) as pool:
    list(pool.map(fetch, urls))       # 4 threads: waits overlap
print("4 threads :", round(time.perf_counter() - t, 2), "s")   # ~2.0 s

Put the two results side by side and the whole chapter resolves into a single sentence. The GIL serializes bytecode execution; it does not serialize kernel waiting. CPU work is bytecode execution, so it serializes — flat line, no speedup. I/O work is kernel waiting with the GIL released, so it overlaps — near-linear speedup until the network or disk saturates. And now the folklore is finally exact: threads win precisely to the degree your program is waiting rather than computing. That is nothing but section one's cpu/wall ratio read a second time. A ratio near 0 (all waiting) means threads overlap it away; a ratio near 1 (all computing) means the GIL turns your threads into a queue.

So threads are a scalpel, not a hammer — useless on compute, decisive on waiting. The last question is the positive one: exactly when do you reach for them, and how do you share results without re-opening the race? →

06When threads still win

The wrong lesson to walk away with is “threads are useless in Python.” They are not; they are specialized. Knowing their exact envelope means you reach for them with confidence, instead of over-engineering an async rewrite or spinning up processes you don't need. Threads win wherever the bottleneck is waiting — which, thanks to the GIL-release-on-blocking rule, is exactly the work the GIL lets overlap. Two canonical patterns cover almost every real case.

The first is I/O concurrency: many blocking calls in flight at once — dozens of HTTP requests, database queries, file reads. Each spends nearly all its time asleep in a kernel wait with the GIL released, so a ThreadPoolExecutor of N workers gives roughly N-way overlap, right up until the network, disk, or remote service saturates. This is the pragmatic, drop-in win. It needs no async rewrite and works with any ordinary blocking library — requests, a database driver, open(). If your code is already written in plain synchronous style and it is I/O-bound, a thread pool is very often the shortest path from slow to fast.

import time
from concurrent.futures import ThreadPoolExecutor

def fetch(url):
    time.sleep(1)                 # stand-in for a real blocking request
    return len(url)

urls = [f"https://example.com/{i}" for i in range(20)]

t = time.perf_counter()
with ThreadPoolExecutor(max_workers=20) as pool:
    results = list(pool.map(fetch, urls))
print("20 requests in", round(time.perf_counter() - t, 2), "s")
# ~1.0 s instead of ~20 s — the twenty waits overlapped

The second is responsiveness: a GUI event loop or a CLI must keep answering input while a slow operation runs. Run the slow blocking call on a worker thread. Because it releases the GIL while it waits, the main thread keeps getting the baton and keeps repainting the window or reading keystrokes — the app stays alive instead of freezing. The classic “UI hangs while a download runs” bug is exactly a blocking call left on the main thread. Moving it to a worker fixes it, precisely because of the release rule you now understand.

import time, threading

def slow_job():
    time.sleep(3)                     # blocking work, off the main thread

worker = threading.Thread(target=slow_job)
worker.start()

while worker.is_alive():           # the 'UI loop' keeps running
    print("still responsive…", flush=True)
    time.sleep(0.5)
worker.join()
print("done")              # the ticks kept printing the whole 3 seconds

Now the honest boundary, because this chapter promised no half-truths. Threads share the heap — that was section 2 — so the moment two threads write the same mutable object, your invariants are back in play, and the GIL will not save them. The GIL protects ob_refcnt. It does not protect your counter, your growing list of results, your “read-then-write.” The disciplined fix is to hand data between threads rather than share it. queue.Queue is thread-safe by construction — workers put results, the main thread gets them — so you never touch one object from two threads at once. When you truly must share, wrap the critical section in a Lock.

import time, queue, threading

results = queue.Queue()          # thread-safe mailbox — no Lock needed

def worker(job):
    time.sleep(0.5)               # the blocking part (GIL released)
    results.put(job * job)         # hand the result back safely

threads = [threading.Thread(target=worker, args=(i,)) for i in range(10)]
for t in threads: t.start()
for t in threads: t.join()

total = 0
while not results.empty():
    total += results.get()
print("collected", total)      # correct every run — no shared mutable object was raced

Finally, close the loop back to section 1's fork, because the whole chapter is one decision tree. Ask the question — faster network or faster CPU? If the answer is network (I/O-bound), threads are your tool for moderate fan-out and any blocking library. For very high fan-out, where thousands of native thread stacks and their context-switch cost become the ceiling, async's cooperative single-thread model wins instead (chapter 27). If the answer is CPU (CPU-bound), threads are the wrong tool entirely. Use processes to get real parallelism, or drop into a C extension that releases the GIL while it computes (chapter 28). Threads are not the general-purpose speed button the folklore sold you. They are the precise instrument for making a program that waits stop waiting one thing at a time — and now you know exactly why.

Threads overlap waits but hit a per-thread memory wall at massive fan-out. Chapter 27 keeps the overlap and removes the wall — one thread, one event loop, thousands of suspended coroutines. That is async. →