62The decision map — pick the right structure on sight
In Chapter 61 we built the thirteenth and last container, Union-Find. With it the volume's shelf is full: thirteen structures across thirteen chapters. This capstone doesn't add a fourteenth. Instead it teaches the one skill the other thirteen were quietly building toward. That skill is how to choose a structure on sight, the way a doctor reads a chart. Here's the plan. First we lay all thirteen on a single master table. Then we draw the flowchart that turns an access pattern into a structure. Then we throw the table away, because the real move isn't memorising it, it's deriving it. The whole way through we keep asking one question that turns thirteen facts into a single reflex: what do I do to this data most, and which shape in memory makes that operation free? By the end you'll meet a fresh problem, name the operation you repeat most, and read the structure straight off it. Then we close the series, from a single bit in Volume 1 to the containers that organise a program's whole world.
base + i×8, one read (chapter 49's row of slots)01Every structure is one answer to one question
Rewind to the promise this volume opened with. A data structure is a deal you strike with memory. You spend a little space here to make an operation instant there. Watch how every chapter signed that same deal differently. The array laid its elements in one contiguous block, so position i is a multiply-add away. That buys O(1) index, and it pays for it with O(n) inserts in the middle. The hash map scattered its entries across a sparse table, so a key becomes an address. That buys O(1) lookup, and it pays with cache misses and empty slots. The heap folded a tree into a flat array so the minimum sits at index 0. And Union-Find, the structure we just left, shrank "are these connected?" to a walk up a parent-pointer forest flattened almost to nothing. Different shapes in RAM, each bending the cost of one operation down to the floor.
So the thirteen aren't thirteen facts to hoard. They're really five shapes memory can take: a row, a grid, a sparse table, a hierarchy, and a web. Each of the thirteen is one of those five with a handful of variations. Hold the five shapes in your head and the whole volume collapses into something you can see.
Let's make the five shapes concrete before we lean on them. The row is the array, the list, and the string, with elements laid end to end. The grid is the 2D array and the matrix, a row of rows. The sparse table is the hash map and the set, mostly-empty slots addressed by a key. The hierarchy is the tree family: the binary search tree, the heap, the trie, and Union-Find's parent forest. The web is the graph, where any node may point at any other. Thirteen names, five shapes. Once you can sort a new structure into one of these five, you already half-know its costs.
Try the sorting on a structure we haven't named yet. A stack — push on top, pop from the top — is just a row you only ever touch at one end, so it inherits the row's O(1) append and needs nothing new. A priority queue that always hands back the smallest item is a hierarchy, a heap wearing a friendlier name. Neither is a fourteenth shape. Once you see the shape underneath, its costs are already half-written, because you met them in the chapter that built that shape.
First, though, let's lay all thirteen side by side — one page you could pin above a desk. Not to memorise, but so the pattern behind it becomes impossible to miss. →
02The master table — thirteen structures on one page
Here is the whole volume as a single lookup, with each row read as a profile. It tells you what shape the structure takes in RAM, what one slot costs, and which operations it makes cheap or expensive. The green cells are what each structure is for, and the red-ish O(n) cells are the price it charges everywhere else. Notice that no structure is green across the board. That's the entire point: every structure is fast at something by being slow at something else.
| Structure | Shape in RAM | Space / item* | Index [i] | Search | Insert | Delete | Min / Max | Built for |
|---|---|---|---|---|---|---|---|---|
| Array (list) | contiguous refs | ~8 B | O(1) | O(n) | O(1)* end · O(n) mid | O(1) end · O(n) mid | O(n) | index by position; iterate in order |
| Linked list | scattered nodes + pointers | ~48 B/node | O(n) | O(n) | O(1) at a node | O(1) at a node | O(n) | splice with no shifting |
| Stack | array / linked, one end | ~8 B | — | O(n) | O(1) push | O(1) pop | O(n) | LIFO — newest first (undo) |
| Queue | ring / linked | ~8 B | — | O(n) | O(1) enqueue | O(1) dequeue | O(n) | FIFO — arrival order |
| Deque | linked fixed blocks | ~8 B | O(n) | O(n) | O(1) both ends | O(1) both ends | O(n) | fast at both ends |
| Hash map (dict) | sparse hash table | ~30 B tbl (~93 all-in) | — | O(n) by value | O(1)* | O(1)* | O(n) | O(1) lookup by key |
| Matrix | flat row-major grid | 1–8 B/cell | O(1) [i][j] | O(n²) | O(1) set | O(1) set | O(n²) | grid by coordinate; dense numbers |
| Binary tree | scattered nodes, 2 pointers | ~56 B/node | O(n) | O(n) | O(1) at a node | O(1) at a node | O(n) | model a hierarchy |
| Balanced BST | ordered nodes | ~56 B/node | O(log n)† | O(log n) | O(log n) | O(log n) | O(log n) | ordered: search + range + sorted walk |
| Heap | flat array, implicit tree | ~8 B | O(1) slot | O(n) | O(log n) push | O(log n) pop | O(1) peek | the min/max, over and over |
| Trie | nodes, child-map per char | heavy / node | O(L) | O(L) | O(L) | O(L) | O(n) | prefix search; autocomplete |
| Graph | adj-list dict / matrix | O(V+E) / O(V²) | — | O(V+E) | O(1) add edge | O(deg) | — | relationships; traversal |
| Union-Find | flat parent array | ~4–8 B | — | O(α) | O(α) union | — | — | connectivity; grouping |
Those space numbers aren't guesses. They're measured. The container's own overhead per element, on CPython 3.12, spans a striking 50× range. It runs from a packed byte for a bytearray, up through the 8-byte reference of a list, to the ~93 all-in bytes a dict spends to buy you O(1) keys.
import sys
from collections import deque
sys.getsizeof([]) # -> 56 empty list = just the header
(sys.getsizeof(list(range(10000))) - 56) / 10000 # -> 8.0 B/item · one 8-byte reference
(sys.getsizeof(deque())) # -> 760 a deque pre-allocates a block
(sys.getsizeof({i:i for i in range(10000)}) - 64)/10000 # -> 29.5 B/item · sparse-table slots only
(sys.getsizeof(set(range(10000))) - 216) / 10000 # -> 52.4 B/item · a set is even sparserLet's read it line by line. An empty list is 56 bytes of header and nothing else. Grow it to ten thousand elements and each one adds exactly 8.0 bytes. That's the contiguous array of references we built back in the array chapter, with over-allocation already amortised in. A deque starts at 760 bytes because it eagerly grabs a whole memory block to make both ends O(1). The dict spends 29.5 bytes per entry on table machinery alone. Add the boxed key and value objects at about ~32 B each for a big int, and it reaches ~93 bytes all-in to record one key→value pair. That's the price of O(1) by key: emptiness. A hash table must stay mostly empty to stay fast.
Why does the table stay partly empty in the first place? Because a hash map only stays O(1) while collisions stay rare, and collisions get common the moment the table fills up. So CPython holds it to about two-thirds full: for every 2 entries it keeps roughly 3 slots, which leaves 1 slot in 3 sitting empty on purpose. Pack it tighter and keys start landing on top of one another, and every lookup slows to a scan. The emptiness isn't sloppiness. It's the headroom the O(1) promise is bought with.
Let's sanity-check one of those numbers by hand. Take a list of 10,000 references: at 8.0 bytes each that's 80,000 bytes for the slots, plus the 56-byte header, so about 80,056 bytes in total. Divide back out and the per-element cost is essentially 8 bytes, exactly the array-of-references picture. The dict tells the opposite story. To keep collisions rare it holds its entries in a table that stays only about two-thirds full, so roughly a third of its slots sit empty on purpose. That deliberate emptiness isn't waste — it is the O(1) lookup you paid for.
Do the same for the dict's all-in figure and it checks out. The 29.5 bytes of table machinery per entry come first. Then the key and the value are separate heap objects, about 32 bytes each for a big int. Add them up: 29.5 + 32 + 32 comes to 93.5, which rounds to the ~93 bytes we quoted. So a single dict entry costs more than eleven bare list slots, and every one of those bytes is buying you the O(1) key lookup.
sys.getsizeof numbers. Contiguous packed data is featherweight; the structures that buy O(1)-by-key or O(1)-splice pay for it in headers, pointers, and deliberate emptiness. Speed and space are the two sides of the same coin — you never get both.assert on the last line is doing real work, too — it refuses to print a row unless all three containers handed back identical answers. (The third row is deliberately slow; that is the finding, not a flaw. The whole script finishes in about fifteen seconds.)import heapq, random, time
from itertools import islice
random.seed(62)
N, Q = 200_000, 200
rows = [(f"p{i:06d}", random.randrange(1_000_000)) for i in range(N)]
spots = [random.randrange(N) for _ in range(Q)] # positions in the stream
names = [rows[i][0] for i in spots] # the SAME records, asked for by name
row = list(rows) # LINEAR - a row, arrival order kept
tbl = dict(rows) # KEYED - a sparse table, name -> score
hp = [(s, i, n) for i, (n, s) in enumerate(rows)] # HIERARCHY - a flat array, min at [0]
heapq.heapify(hp)
def ms(f):
t = time.perf_counter(); r = f(); return (time.perf_counter() - t) * 1e3, r
# VERB 1 - replay: hand me the i-th event that arrived
def v1_row(): return [row[i][1] for i in spots] # base + i*8
def v1_tbl(): return [next(islice(tbl.values(), i, None)) for i in spots] # walk i entries
def v1_hp(): return [next(s for s, j, _ in hp if j == i) for i in spots] # scan for the index
# VERB 2 - fetch one player's score by name
def v2_row(): return [next(s for n, s in row if n == a) for a in names]
def v2_tbl(): return [tbl[a] for a in names]
def v2_hp(): return [next(s for s, _, n in hp if n == a) for a in names]
# VERB 3 - take the lowest score, over and over, removing it each time
def v3_row():
c = list(row)
return [c.pop(min(range(len(c)), key=lambda k: c[k][1]))[1] for _ in range(Q)]
def v3_tbl():
c = dict(tbl)
return [c.pop(min(c, key=c.get)) for _ in range(Q)]
def v3_hp():
c = list(hp)
return [heapq.heappop(c)[0] for _ in range(Q)]
def show(x): return f"{x:,.3f} ms" if x < 10 else f"{x:,.1f} ms"
print(f"one dataset - {N:,} (player, score) records - three verbs - {Q} calls each\n")
print(f"{'the verb your hot loop repeats':<32}{'list':>12}{'dict':>12}{'heap':>12} fastest")
for name, fr, ft, fh in [("replay: the i-th event", v1_row, v1_tbl, v1_hp),
("fetch a score by name", v2_row, v2_tbl, v2_hp),
("take the lowest, over and over", v3_row, v3_tbl, v3_hp)]:
tr, ar = ms(fr); tt, at = ms(ft); th, ah = ms(fh)
assert ar == at == ah # identical answers, every time
t = {"list": tr, "dict": tt, "heap": th}
print(f"{name:<32}{show(tr):>12}{show(tt):>12}{show(th):>12} {min(t, key=t.get)}")one dataset - 200,000 (player, score) records - three verbs - 200 calls each the verb your hot loop repeats list dict heap fastest replay: the i-th event 0.041 ms 48.7 ms 621.3 ms list fetch a score by name 458.9 ms 0.044 ms 683.2 ms dict take the lowest, over and over 6,596.2 ms 5,780.4 ms 6.039 ms heap
A table is a lookup, and lookups are for people who already know what they want. The harder skill is arriving with a vague problem and walking out with a structure. That's a flowchart — and here it is, made clickable. →
03The decision flowchart — from access pattern to structure
Forget the thirteen names for a moment and ask a single question about your data: what do I do to it most? Not occasionally, but most, in the hot loop, a million times over. Every branch below is a different answer to that one question. Each branch lands on the structure whose layout makes that answer O(1), or as close as the problem allows. Read it top-down. Identify your dominant access pattern, follow the branch, and arrive at the container.
Here's what "dominant access pattern" means in the flesh. Say you're counting how often each word appears in a book. The operation you do most is "take this word and bump its count," millions of times over. That's a lookup by key, so the layout that turns a key into an address wins, and you reach for a dict. Now say you're instead always asking "what's the smallest unfinished task?" That's remove-the-minimum, over and over, so the layout that keeps the minimum at the front wins, and you reach for a heap. Same reasoning, two problems, two shapes. You never listed all thirteen. You named the hot operation and let it point.
Two branches we skipped, just to complete the five. Say the hot operation is "what's at row r, column c" on a game board. That's access by coordinate, and a grid answers it with one multiply-add: on a 100-wide board, cell (3, 4) sits at offset (3·100 + 4)·8 = 2432 bytes from the base, read in a single step. Or say the hot operation is "who is connected to whom" — friends, links, roads. That's a web, and a graph stores each node's neighbours right beside it. Five access patterns, five shapes. Name the operation and the shape is already chosen.
One more, because the reflex only sticks with reps. Say you're serving autocomplete, and the hot operation is "find every word that starts with ca." That's a prefix query. The layout that stores one node per letter and shares common stems wins, so you reach for a trie. Now change the question to "is this exact word in my spellcheck list?" The prefix stops mattering, plain membership wins, and a set answers in O(1). Same words, two questions, two shapes. The data never told you which structure to use. The operation did.
set is just a dict that dropped its values. Both turn the thing you are holding into an address. Both pay for it in deliberate emptiness: about a third of the slots sit unused so collisions stay rare.maxlen set, an append at the full end silently drops the far end, which is a ring buffer in one argument: the last 200 messages, the last 60 frames, the last hour of samples. No if len(...), no shifting.heapq.nlargest(k, xs) when you only need it once..sort(). Cheap while the list is modest; past a few tens of thousands the memmove starts to bite and you want a real ordered tree.functools.lru_cache is. Name each hot verb, take the layout that makes it free, and stitch.INPUTfrom collections import deque
# one suitcase, one Sunday night, five verbs
pocket = {"passport": "LM4820913", "boarding pass": "NZ0148 14C"} # by KEY
days = ["linen shirt", "tee + shorts", "swimsuit", "smart shirt"] # by POSITION
door = [] # LIFO
gate = deque(["A17", "A18", "A19"]) # FIFO
packed = {"socks", "toothbrush", "charger"} # MEMBERSHIP
for last_minute in ("charger", "keys", "sunglasses"):
door.append(last_minute) # thrown on the pile, top last
print("passport ->", pocket["passport"]) # dict - one key, no scan
print("day 3 outfit ->", days[2]) # list - one address
print("grab on the way ->", door.pop()) # stack - newest first
print(" then->", door.pop())
print("boarding next ->", gate.popleft()) # queue - oldest first
print(" then->", gate.popleft())
print("socks packed? ->", "socks" in packed) # set - one probe
print("towel packed? ->", "towel" in packed)
print("still by door ->", door, "| still queued ->", list(gate))
print()
verbs = [("by a key", "dict", "hash the key, land on the slot"),
("by position", "list", "base + i*8, one read"),
("newest first", "stack", "append / pop, one end only"),
("oldest first", "deque", "popleft, no shifting"),
("is it in there?", "set", "hash the value, look once")]
for verb, structure, why in verbs:
print(f" {verb:16} -> {structure:6} : {why}")OUTPUTpassport -> LM4820913
day 3 outfit -> swimsuit
grab on the way -> sunglasses
then-> keys
boarding next -> A17
then-> A18
socks packed? -> True
towel packed? -> False
still by door -> ['charger'] | still queued -> ['A19']
by a key -> dict : hash the key, land on the slot
by position -> list : base + i*8, one read
newest first -> stack : append / pop, one end only
oldest first -> deque : popleft, no shifting
is it in there? -> set : hash the value, look once- A list used as a queue. The most common of all three, because
.pop(0)reads so innocently — and it shifts every remaining element down one slot. Measured here, 2,000 removals from the front of a 300,000-element container:list.pop(0)took 0.3175 s,deque.popleft()took 0.000047 s — about 6,769×. The fix is one import (chapter 52). - A list asked “is it in there?” This one changes the exponent, not the constant: inside a loop, an O(n) membership test turns an O(n) job into O(n²). Measured, 1,000 worst-case checks against 300,000 items:
x in listtook 2.0501 s,x in settook 0.000023 s — about 87,600×. The fix is one character:[becomes{(chapter 54). - Re-sorting after every insert.
appendthen.sort()is correct and feels tidy, and it re-does work you already had. Measured: building a sorted list of 2,000 random values cost 0.00290 s that way against 0.00035 s withbisect.insort— 8.2×. At 20,000 values it was 0.32105 s against 0.01893 s, 17.0×. Note the ratio grew: this is not a constant factor you can shrug off, it widens with your data (chapter 57).
Now make it yours. Below, toggle the operation you'd do most and the flowchart hands back the structure, along with the one-line reason its layout wins. There's no "best" button, because there is no best structure — only the one whose shape makes your hot operation free.
The flowchart tells you which structure. But to trust it, you have to feel why the same operation can be free in one and ruinous in another. Pick an operation and watch all thirteen sort themselves. →
04Same operation, every structure — watch the costs sort out
Here's the claim the whole table rests on: costs aren't assigned, they're forced by the layout. "Get the item at position i" is O(1) on an array because the address is pure arithmetic — you compute base + i·8 and read it in one step. The same operation is O(n) on a linked list because position isn't stored anywhere, so the only way to reach the i-th node is to start at the head and follow next i times. Same operation, same name, two layouts. The layout, not the operation, decides the price.
Put real numbers on that gap. Suppose both structures hold a million items and you want the last one. On the array you compute one address and read it, a single step no matter how big the array grows. On the linked list you follow next from the head, so reaching the millionth node means a million pointer-hops. One step versus a million, and nothing changed but the shape the data lives in. That is what forced by the layout means. The verb is identical, and the price is set purely by how you can reach the data.
Play with it and the punchline lands in a single sweep. The array wins "index i" but loses "insert at front". The hash map wins "by key" and can't even offer "position i". The heap owns "remove the minimum" and is useless for membership. Every winner is somebody's loser, so there is no row that's green everywhere. Choosing a structure is choosing which column you want green and which you can afford to leave red.
Put a number on "loses insert at front." Take an array of a million items and push one new element onto the front. Every existing item has to slide up one slot to make room, so that's a million writes for a single insert. A hash map asked to store one new key does one write and is done. A million versus one, the same lopsided gap as before, and again nothing changed but the shape the data lives in. The array put position at your fingertips, and the bill for that comes due the instant you disturb position.
And that isn't bad luck you could engineer away — it's forced. A layout can only put one thing at your fingertips: the array puts position there, the hash map puts the key there, the heap puts the minimum there. Optimising for one arrangement of bytes necessarily de-optimises the others, because the bytes can only sit in one arrangement at a time. That's why the green cell always drags a red cell behind it. There is no free lunch in memory, only a choice of which meal you're buying.
next is a jump to an unrelated address the prefetcher can't guess. Measured here: summing a contiguous array('q') of a million ints versus walking a million-node linked chain — same O(n) — the chain ran about 2.4× slower (machine-dependent). When two structures tie on paper, the contiguous one usually wins on the metal. That's why a heap is a flat array and not a pointer tree.set is a thousand. Measured: a worst-case x in list over 100k elements took ~0.64 s for 1000 checks; x in set did 100× more checks in ~0.003 s. Same intent, different structure, four orders of magnitude. This is the single most common real-world performance bug — and it's a data-structure bug, not an algorithm one.import timeit
# SAME job — remove from the front 2000 times — on two layouts:
timeit.timeit("q.pop(0)", "q=list(range(100000))", number=2000) # -> ~0.33 s list: O(n) shift
timeit.timeit("q.popleft()", "from collections import deque;"
"q=deque(range(300000))", number=2000) # -> ~5e-5 s deque: O(1)
# SAME question — is it present? — scanned vs hashed:
timeit.timeit("99999 in c", "c=list(range(100000))", number=1000) # -> ~0.64 s list: O(n)
timeit.timeit("99999 in c", "c=set(range(100000))", number=100000) # -> ~0.003 s set: O(1)Line by line, and every number here was RAN. Popping the front of a list is O(n) because every remaining element shifts down one slot. That costs ~0.33 s for 2000 pops. A deque does the same removal in O(1) by moving a pointer, not the data. That's about 6000× faster on this machine. Then membership. Scanning a list for a value is O(n), and 1000 worst-case scans of 100k elements cost ~0.64 s. A set hashes straight to the answer, so a hundred times as many checks finish in ~0.003 s. Same goals, different layouts, and the gap is the difference between a snappy app and a hung one. (Timings are machine-dependent, but the ratios are the point.)
You've now seen the table, the flowchart, and the costs. Time to throw the table away — because the people who are genuinely good at this don't carry it in their heads at all. →
05You don't memorise the table — you derive it
Here's the 1% move, and it's smaller than you'd think. Faced with any problem, the expert doesn't scan a mental list of thirteen structures. They ask one question: "what's the operation I'll do most, and which layout makes that operation O(1)?" Answer it, and the structure stops being a choice you agonise over. It becomes a consequence you read off. Need constant lookup by an id? A layout that turns a key into an address → hash map. Need the next-smallest thing again and again? A layout that keeps the minimum at the front → heap. Suggesting words as someone types? A layout with one node per letter → trie. The structure falls out of the operation.
The deeper cut — the best structures are often two structures welded together
Real systems rarely reach for one container. When a problem has two hot operations, you combine the two layouts that each make one of them O(1):
The classic example is the LRU cache, the little memory that remembers your most recent lookups and forgets the stalest. It has two hot operations. "Find the value for this key" wants a hash map, the layout that makes lookup O(1). "Move this item to most-recent and drop the oldest" wants a doubly linked list, the layout that makes splicing a node O(1). Neither shape can do both jobs alone. So you run them together: the hash map's value is a pointer into the list, and every operation stays O(1). Two blind spots, each covered by the other's strength.
Trace a single get(key) to see both layouts fire at once. First the hash map turns the key into an address and hands you the node in O(1). That node already knows its neighbours in the list, so you unhook it and re-link it at the most-recent end, again O(1), with no scanning to find it. One call, two shapes, each doing the half the other can't. Drop either one and a step that was O(1) collapses to O(n): without the hash map you'd search the list for the key, and without the list you'd have no cheap way to reorder by recency.
- LRU cache — needs O(1) "look up by key" and O(1) "move this to most-recently-used." Answer: a hash map (key → node) welded to a doubly linked list (recency order). Each structure covers the other's weakness; that pairing is Python's own
functools.lru_cacheand every database page cache. - Priority queue with updates (Dijkstra, Vol 3) — a heap for O(log n) "next smallest," plus a hash map from node to its heap position so you can O(1)-find an entry to decrease its key.
- Inverted index (every search engine) — a hash map from word to a sorted array of document ids; the map finds the word, the array supports fast intersection.
The move scales cleanly. Name each hot operation, pick the layout that makes it O(1), then stitch them together. "Good at data structures" at the highest level is really composition — it means knowing which two shapes cover each other's blind spots.
Here's a second pairing, so the pattern is unmistakable. Suppose you must report the running median of a stream of numbers that never stops arriving. Keep the smaller half in a max-heap and the larger half in a min-heap, balanced so their sizes differ by at most one. The median then always sits at one of the two roots, so you read it in O(1) and absorb each new number in O(log n). No single container gives you that; two heaps facing opposite directions do. That's composition again: two shapes, each covering the other's blind spot.
Myth
There's a "best" data structure, so learn the fastest one and default to it. Lists are simple, so use lists for everything, and dicts are powerful, so reach for dicts every time.
Reality
There is no fastest, only fittest for an access pattern. Every structure is fast at one thing by being slow at others. The skill isn't a favourite. It's the reflex to name your dominant operation, pick the layout that makes it free, and accept the costs elsewhere.
One honest caveat before we zoom out. These O() labels describe how cost grows, not the cost at tiny sizes. For a handful of items a plain list often beats a dict, because scanning five slots is faster than hashing a key and chasing a pointer. The access-pattern reflex still holds. It just earns its keep once the data gets big, which in the hot loop is exactly when it matters. So pick the fit for the size you'll actually run at, not the size on the slide.
How small is "small"? On CPython, scanning a list for a value walks items a cache line at a time, which is blisteringly fast per step. A dict instead pays a fixed toll up front: hash the key, jump to a slot, chase a pointer, compare. For a handful of items that toll dominates, so the list wins. Somewhere around a few dozen entries the O(n) scan finally overtakes the flat O(1) hash, and the dict pulls ahead for good. You don't need the exact crossover. You need the instinct to distrust Big-O at tiny sizes and trust it completely once the loop runs hot.
One question — "what do I do most, and which layout makes it O(1)?" — is the entire volume, distilled. Which means you're ready for the last section: where a person goes once the containers stop being mysterious. →
06Where you go from here — a bit, a structure, a system
Step back and look at the whole climb. Volume 1 started at a single bit, a box in RAM that's 0 or 1. It built up through bytes, addresses, references, and the heap, until a Python object was something you could picture down to its header. Volume 2 made that machine program: files, modules, tests, concurrency. Volume 3 taught it to think: Big-O, the memory hierarchy, divide-and-conquer, greedy, DP, and the algorithms that turn brute force into elegance. And this volume, Volume 4, gave those algorithms their containers. Those are the thirteen shapes that decide, before a single line of logic runs, what's cheap and what's ruinous. Bit → byte → object → structure. You can now trace a program's data from the metal all the way up to the shape it lives in.
So where next? Outward, into systems built from exactly these parts. Open any database and you'll find our structures load-bearing. An index is a B-tree, a fat, disk-friendly BST, or else a hash index. Write-optimised stores like Cassandra and RocksDB layer LSM-trees over sorted arrays and heaps, and a query planner walks a DAG. Step up to distributed systems and the same shapes reappear, hardened for scale. Consistent hashing is a hash map wrapped around a ring, so machines can join and leave without reshuffling the world. Merkle trees, which are hash trees, let Git, Bitcoin, and Amazon's Dynamo verify terabytes by comparing a single root hash. Bloom filters trade a little accuracy for a set that fits in a breath of memory. CRDTs let two offline copies of a structure merge without conflict. None of these is new magic. Each is one of your thirteen, chosen because its layout makes the system's hottest operation cheap. That's the exact reasoning you now own.
add_score(player, score) arrives constantly — a run finishes somewhere in the world and posts a number, and only a personal best changes anything. top(k) is what the front page draws: the five names at the summit. rank(player) is the line every player sees on their own screen — you are #7,868 of 7,955 — and it is the most-called of the three, because everybody wants their own number and almost nobody is in the top five. You will meet the small cousin of this in the drill at the end of the chapter — a board that is only ever asked for the top three, which is one verb, therefore one structure, settled in two lines. Give that same product two more verbs and the answer stops being a container at all. Three labels before we start, so this reads as what it is. One: nothing below is a new structure. Every part of the answer is something you built in this volume, and if a fourteenth container appears, you have been misled. Two: the destination is not a secret, and this chapter has already handed it to you twice. The toolbelt card told you in one line that two hot verbs mean two layouts, and the deeper-cut fold in the previous section said it outright — name each hot operation, pick the layout that makes it O(1), then stitch them together, which is what being good at data structures amounts to once you are past the table. Nothing below improves on that sentence. What a sentence cannot hand you is the part that actually decides a design — which single-container answer a competent person writes first, what that costs on a clock, and which verb quietly refuses to be served by the others. Three: every version below is exact. The run checks each of them against a deliberately stupid referee at a thousand separate checkpoints, not merely at the end.player → best makes add_score a single write and answers what is my score in one probe; when top(5) or rank comes in, sort the items, or count how many beat you. A sorted array of (score, player), held in order by bisect.insort, makes top(k) a slice off the end and rank a binary search — the two hard verbs, both cheap, which is why this is the design that feels cleverest. A heap parks the champion at index 0 for free. Each is a genuinely good answer to the verb it was built for. Now watch which verb each one cannot serve at all.top(10) and rank served 1,000 times along the way. The dict, re-sorting on every top(): 1,107.0 ms. The sorted array, which keeps no by-name view and so must scan the whole array just to find the old entry it is replacing: 1,735.8 ms. The heap, which keeps no by-name view either and cannot express rank without first rebuilding a dict of everyone's best: 9,990.2 ms. Composing the three instead of choosing between them: 70.2 ms — and all four agreed on every one of the 1,000 top-tens and all 1,000 ranks, so this is a speed argument and not a correctness one. Then take it to a size worth having: ten times the stream, 200,000 runs over 50,000 players, 2,000 reports. The dict-and-sort design took 154,569.2 ms — two and a half minutes of a machine sorting fifty thousand players to print five names, over and over, because it had no other way to find them. The composed one took 3,098.7 ms. And look at what the ratio did between the two runs: 15.8×, then 49.9×. Those two figures are this machine's and will not be yours; the thing they are doing will be. It grew. That is not a constant factor you can shrug off; that is the exponent, doing what exponents do.by name hires the sparse table. the top few hires a heap sized by k rather than by n, so the size of the crowd never enters its bill at all. where do I stand hires ordered storage. Nobody arbitrates between them, because there was never one job to arbitrate over. And the recognition comes with one honest bill. Ordered storage and hashed storage pull in opposite directions: a hash table scatters keys on purpose so that a lookup is one probe, and scattering is precisely what destroys order — so the rank tier cannot borrow the dict's work, it has to keep a second, ordered copy, and every personal best now pays a memmove through it. The toolbelt's bisect.insort card warned you about exactly that memmove, and handed you the threshold in words: cheap while the list is modest, and past a few tens of thousands you want a real ordered tree. Here is that same warning with a clock on it — 200,000 posts with the rank tier switched off and then on, measured here, so read the climb rather than the milliseconds: at 10,000 players, 23.7 ms against 335.2 ms; at 86,393 players, 72.5 ms against 4,161.1 ms; at 181,028 players, 364.2 ms against 10,662.7 ms. The dict-and-heap half hardly notices the crowd. The exact-live-rank half does not scale, and saying otherwise would be the one dishonest sentence in this whole book. Real boards do one of two things about it, and you already know enough to pick. Either they rebuild the ranking on a cadence — sort every best once a minute, serve a rank up to a minute stale, which no player has ever noticed on a screen — or they replace the array with chapter 57's balanced search tree carrying a subtree count in each node, so insert and rank are both O(log n). That second one is the order-statistic variant already hiding in this chapter's own fine print, and it is not a fourteenth shape: it is your hierarchy, wearing one extra integer per node.import heapq, random
from bisect import bisect_left, bisect_right, insort
class Leaderboard:
"""Three verbs, three layouts. Not one of them doing another's job."""
def __init__(self, k=5):
self.k = k
self.best = {} # dict player -> best score | verb: BY NAME O(1)
self.on = set() # set who is on the board | verb: IS IT IN? O(1)
self.board = [] # size-k heap the k leaders, min[0] | verb: THE TOP FEW O(log k)
self.order = [] # sorted array every best score | verb: RANK O(log n)
def add_score(self, player, score):
old = self.best.get(player)
if old is not None and score <= old:
return False # not a personal best
if old is not None:
self.order.pop(bisect_left(self.order, old)) # retire the superseded best
self.best[player] = score
insort(self.order, score) # sorted at every instant
if player in self.on: # already a leader: refresh the k-heap
self.board = [(self.best[q], q) for _, q in self.board]
heapq.heapify(self.board) # O(k) - k is 5, never n
else:
heapq.heappush(self.board, (score, player))
self.on.add(player)
if len(self.board) > self.k: # the weakest keeper falls off
self.on.discard(heapq.heappop(self.board)[1])
return True
def score(self, player): # BY NAME - one probe
return self.best[player]
def top(self, k=None): # THE TOP FEW - touches k, never n
return [(p, s) for s, p in sorted(self.board, key=lambda e: (-e[0], e[1]))[:k or self.k]]
def rank(self, player): # RANK - one binary search
s = self.best[player]
return len(self.order) - bisect_right(self.order, s) + 1
random.seed(62)
board, me = Leaderboard(k=5), "p03127"
players = [f"p{i:05d}" for i in range(8_000)]
for n in range(1, 40_001): # an evening on the game
board.add_score(random.choice(players), random.randrange(1, 1_000_000))
if n in (10_000, 40_000):
print(f"after {n:,} runs posted by {len(board.best):,} players")
for i, (p, s) in enumerate(board.top(), 1):
print(f" {i}. {p} {s:>7,}")
print(f" {me}: best {board.score(me):>7,}"
f" rank {board.rank(me):>5,} of {len(board.best):,}\n")
brute = 1 + sum(1 for v in board.best.values() if v > board.best[me])
print(f"rank check - O(log n) binary search {board.rank(me):,}"
f" vs the obvious O(n) count {brute:,} -> {board.rank(me) == brute}\n")
for verb, layout, cost in [("what is my score?", "dict ", "O(1)"),
("who are the top 5?", "size-5 heap ", "O(k log k)"),
("where do I stand?", "sorted array ", "O(log n)")]:
print(f" {verb:<19} -> {layout} {cost}")
print("\nthree verbs, three hires. the composition IS the decision map.")after 10,000 runs posted by 5,700 players 1. p05089 999,794 2. p03579 999,732 3. p00626 999,635 4. p04400 999,527 5. p05631 999,198 p03127: best 201,518 rank 5,063 of 5,700 after 40,000 runs posted by 7,955 players 1. p02601 999,906 2. p03540 999,852 3. p02193 999,801 4. p05089 999,794 5. p00739 999,774 p03127: best 201,518 rank 7,868 of 7,955 rank check - O(log n) binary search 7,868 vs the obvious O(n) count 7,868 -> True what is my score? -> dict O(1) who are the top 5? -> size-5 heap O(k log k) where do I stand? -> sorted array O(log n) three verbs, three hires. the composition IS the decision map.
p03127 in the output: the same 201,518 at both checkpoints, and a rank that slid from 5,063 to 7,868 without them touching the game once. Their number never moved; the field moved around them. That is the whole reason rank is the expensive verb — it is a question about everyone, asked one player at a time — and it is why it needs a layout the other two would never have paid for. Which is the thesis of this book, said once more, plainly: you do not pick a container. You name what you do to the data most, take the shape that makes it free, and you decide, with your eyes open, what you are willing to pay everywhere else.And that's the quiet gift of this whole series. You didn't memorise thirteen containers or two dozen algorithms. You learned to look at a problem and see the shape it wants: the operation underneath, the layout that makes it free, and the price you're willing to pay elsewhere. That reflex doesn't expire. It's the same move whether you're picking a Python dict, designing a database index, or sketching a system on a whiteboard for a million users. Start from a bit. Ask what you do most. Choose the shape that makes it O(1). You began this series not knowing what a variable really was. You're leaving it able to reason about the machine from the transistor to the distributed cluster, from first principles, all the way up. Go build something, and pick your structures on sight.
1. A chat window must show the last three messages, and only those, as new ones keep arriving. · 2. A leaderboard takes scores all day but is only ever asked for the top three. · 3. A spellchecker holds a hundred thousand words and is asked is this a word? a million times a minute. · 4. A booking form must suggest city names the moment somebody has typed
we. · 5. A moderation tool receives reports pairing duplicate accounts, one at a time, and must answer are these two the same person? at any moment.The check. Every answer should be one import or none, and no answer should need a loop over everything you are holding. If any of the five made you write a scan, you named the wrong verb — go back and say out loud what the hot loop does, in five words, before you choose again. Then open the solution: it is the same five, worked, and the last few lines of the book.
show the solution
from collections import deque
import heapq
# 1 - "keep only the last 3 messages" -> newest in, oldest falls out -> deque(maxlen)
chat = deque(maxlen=3)
for msg in ["hey", "you up?", "landed", "bag lost", "found it"]:
chat.append(msg)
print("1 last 3 messages ->", list(chat), " (deque, ch52 - the ends are the verb)")
# 2 - "top 3 scores, inserted constantly" -> always-max -> a size-k heap
top = []
for score in [412, 980, 77, 1503, 640, 1201, 88]:
heapq.heappush(top, score)
if len(top) > 3:
heapq.heappop(top) # evict the smallest of the keepers
print("2 top 3 scores ->", sorted(top, reverse=True),
" (heapq, ch58 - the root IS the answer)")
# 3 - "is this a word?" a million times -> membership -> set
words = {"the", "graph", "python", "metal", "structure"}
print("3 'python' a word? ->", "python" in words,
"| 'pythom'?", "pythom" in words, " (set, ch54 - one probe, any size)")
# 4 - "suggest as they type 'we'" -> prefix -> trie (a dict of dicts)
trie = {}
for city in ["wellington", "westport", "whanganui", "auckland"]:
node = trie
for ch in city:
node = node.setdefault(ch, {})
node["$"] = True
def suggest(trie, prefix):
node = trie
for ch in prefix:
node = node.get(ch)
if node is None:
return []
out = []
def walk(n, sofar):
if "$" in n:
out.append(sofar)
for ch, kid in n.items():
if ch != "$":
walk(kid, sofar + ch)
walk(node, prefix)
return out
print("4 suggest('we') ->", suggest(trie, "we"),
" (trie, ch59 - stand on the prefix, read below)")
# 5 - "are these two accounts the same person?" -> merge + ask -> union-find
parent = {}
def find(x):
parent.setdefault(x, x)
root = x
while parent[root] != root:
root = parent[root]
while parent[x] != root:
parent[x], x = root, parent[x]
return root
def union(a, b):
ra, rb = find(a), find(b)
if ra != rb:
parent[rb] = ra
for a, b in [("u1", "u7"), ("u7", "u9"), ("u2", "u5")]:
union(a, b)
print("5 u1 and u9 the same? ->", find("u1") == find("u9"),
"| u1 and u2?", find("u1") == find("u2"),
" (union-find, ch61 - forget the route, keep the label)")
print("\nfive problems, five verbs, zero memorising:")
for verb, pick in [("both ends, bounded", "deque"), ("always the max", "heap"),
("is it in there?", "set"), ("what starts with...", "trie"),
("same group?", "union-find")]:
print(f" {verb:20} -> {pick}")
print("\nyou never chose a structure. you named the verb, and it chose itself.")
# ---------------- OUTPUT, verbatim (CPython 3.12.7) ----------------
# 1 last 3 messages -> ['landed', 'bag lost', 'found it'] (deque, ch52 - the ends are the verb)
# 2 top 3 scores -> [1503, 1201, 980] (heapq, ch58 - the root IS the answer)
# 3 'python' a word? -> True | 'pythom'? False (set, ch54 - one probe, any size)
# 4 suggest('we') -> ['wellington', 'westport'] (trie, ch59 - stand on the prefix, read below)
# 5 u1 and u9 the same? -> True | u1 and u2? False (union-find, ch61 - forget the route, keep the label)
#
# five problems, five verbs, zero memorising:
# both ends, bounded -> deque
# always the max -> heap
# is it in there? -> set
# what starts with... -> trie
# same group? -> union-find
#
# you never chose a structure. you named the verb, and it chose itself.# ------------------------------------------------------------------ # 1 last 3, newest in / oldest out ..... deque(maxlen=3) ch52-53 # 2 top 3, inserted all day ............ a size-3 heap ch58 # 3 "is this a word?", a million times . set ch54 # 4 "what starts with we?" ............. trie: dict-of-dicts ch59 # 5 merge pairs, then "same person?" ... union-find ch61 # # And that is the end of the book. Sixty-two chapters ago you were not # sure what a variable really was. You are leaving able to hear the verb # inside a messy problem and reach, without looking, for the shape that # makes that verb free. Nobody can take that back off you. # # Close the tab. Open an editor. Go find a problem worth the reflex. # ------------------------------------------------------------------
This closes Volume 4 — and the series. Your next move isn't another chapter here; it's a real system. Take the reflex you built and open the door marked databases and distributed systems: B-trees on disk, LSM-trees under your writes, hash rings across a fleet, Merkle trees keeping it all honest. Same thirteen shapes, bigger stage. You're ready. →
- A name is not a box; it is a label tied to an object. Everything you have ever written down in Python is an object sitting at an address behind a header, and
b = aties a second label to the same one rather than copying it — which is why two names can watch the same list change underneath them, and why that stopped being spooky the moment you could picture the arrow. - Nothing above the metal is free — it is all paid for in bytes. A bit is 0 or 1; a Python object carries about 28 bytes of header before it holds anything; a list slot is one 8-byte reference; a dict spends roughly 93 bytes per entry and keeps about a third of its slots empty on purpose, because that deliberate emptiness is the O(1) you were sold. Convenience is never free. It is invoiced somewhere, and you now know where to look.
- Measure; do not guess. A clock and a failing test outrank every opinion in the room, including your own — which is how you found out that two loops with the same O(n) can differ by 2.4×, one streaming a cache line while the other chases pointers no prefetcher can predict. Big-O tells you the shape of the curve; only a stopwatch tells you where you are standing on it.
- Growth decides, and the cheapest work is work you already did. A faster machine buys you a constant; it never buys you back an exponent, which will find you the week your data doubles — and the deepest move in the algorithms volume was always the same one in different clothes: store the answer instead of re-deriving it. Memoise, tabulate, keep a running count.
- The verb chooses the structure. Name the one operation your hot loop repeats most, take the layout that makes exactly that free, and accept the bill everywhere else with your eyes open — and when a problem turns out to have two hot verbs, do not hunt for a container that does both. Hire two layouts and weld them at the seam.
Thirteen structures, five shapes, one question — here each container is built from bare lists and dicts so you can watch the layout itself decide what is free and what is ruinous.