53The deque — fast at both ends
In Chapter 52 we built the queue — first in, first out. It was cheap at both ends only because we picked the right structure underneath. Here we meet the container that refuses to choose an end at all. A deque — say "deck", short for double-ended queue — is a stack and a queue fused into one. You push and pop at the front and the back, and all four of those moves cost O(1). Here's the plan. We'll open up the memory to find the one trick that buys both-ends speed: a chain of small blocks. Then we'll find the single caveat that tells you when not to reach for it. And the whole way through we keep asking the thing that separates a tool from a toy — what access pattern makes a deque exactly right, and where does that same shape quietly betray you? By the end you'll spot an "ends-only" pattern on sight. You'll use a bounded deque as a self-trimming history, and collapse a sliding-window problem from O(n·k) down to O(n).
Where does refusing to choose an end actually pay off? Picture a print spooler feeding one printer. Ordinary documents line up at the back, and the printer pulls the next job off the front. That's a plain queue. But now a rush job arrives, and it has to jump the line. With a deque you appendleft it onto the front, and the printer grabs it next. Same structure, both ends live: normal work flows in the back, urgent work cuts in at the front, and the worker always pulls from the front. A work-stealing scheduler leans on the same trick. A thread takes tasks off its own front, while idle threads steal from its back. Any time "add here, remove there" points at two different ends, you want a deque.
q.append(song) · O(1)q.appendleft(song) · O(1) — the move a list cannot make cheaplyq.popleft() · O(1)q.pop() · O(1) at the other enddeque(maxlen=50) · the far end falls off for free01Both ends, for free
Let's start with the list you already know, because the deque is a direct answer to something the list does badly. From Volume 1, a list is a contiguous block of 8-byte references. That makes it cheap at the back. An append is amortized O(1), because it just drops a reference into a spare slot. But the front is a trap. pop(0) and insert(0, x) have to shift every remaining reference one slot over to keep the block gap-free. That is the front-shift we dissected when we first took the list apart. Watch what it costs. I timed it: draining 50,000 items one at a time with list.pop(0) took about 4.7 seconds. The same drain with a deque took about 4 milliseconds. That is roughly a thousandfold gap, and it widens with n because the list is quietly O(n²).
Where does that O(n²) actually come from? Draining from the front, the first pop(0) shifts the other 49,999 references down a slot, the next shifts 49,998, and so on down to a final lonely shift. Add that staircase up: 50,000 × 49,999 ÷ 2 ≈ 1.25 billion reference moves for a single drain. The deque does the same job in 50,000 cursor steps, one per item. That is why the gap is not a constant factor but a widening canyon.
append at the back, pop(0) at the front — looks innocent and is quietly quadratic: each pop(0) shifts all remaining references one slot, so serving n items costs about n²/2 moves. That's the ~4.7 s I measured draining 50,000 items, versus ~4 ms on a deque. It is one of the most common reasons a Python service that "worked in testing" seizes up under load. If items leave from the front, reach for a deque.A deque is engineered so both ends are as cheap as a list's back: O(1) at the front and the back alike. The whole secret is the layout. So let's look at the memory before we look at the code.
append or appendleft is one reference write; a fresh block is hooked on only when an end block fills.
pop(0) slides all survivors left) but a deque exactly N. That's the O(n²) trap from the danger box above — push N past a million and a one-line queue quietly asks the CPU for half a trillion moves.Why 64? Why blocks at all — why not one node per item like a textbook linked list, or one big array like a list? →
02Link blocks, not items
Here's the design insight, and it's the kind of "aha" worth stealing. A textbook doubly-linked list gives you O(1) ends, but it pays a whole node object per item. I measured a minimal Python node, a class with __slots__ for value/next/prev. Each one weighs 56 bytes, and ~344 bytes if it carries an ordinary __dict__. Worse, every node is a separate heap allocation scattered across RAM. Walking such a list is pure pointer-chasing, and each hop is a fresh cache miss — the pointer-chasing tax we clocked back in Volume 3. A plain array is the opposite. Its 64 references sit shoulder to shoulder and stream through cache. But the front is O(n), and a growth means reallocating the whole block.
Why does that scattering matter so much? The CPU doesn't read one byte from RAM at a time. It pulls a whole cache line, 64 bytes, into fast on-chip memory and hopes you'll use the neighbours next. When your data sits shoulder to shoulder, the next item is already in cache, and the read costs a few CPU cycles. When each item is a separate heap allocation somewhere random in RAM, the neighbour is useless, and the CPU stalls waiting on main memory, on the order of a hundred cycles for the miss. That's the pointer-chasing tax: not the pointer hop itself, but the stall behind it, paid on every single node. Multiply it across a million items and the node-per-item list loses badly, even though its big-O looks identical.
The deque's author refused to choose. Link the blocks, not the items. Inside one block, 64 references are contiguous — one allocation feeds 64 items, and they scan cache-friendly. Between blocks, prev/next pointers give the front the exact same cheap, local growth the back already had. You get the linked list's flexible ends and keep most of the array's locality.
Let's make the layout concrete with a count. Each block holds 64 references, so a deque of 200 items spans about ⌈200 ÷ 64⌉ = 4 blocks. Four blocks means only three links between them, so reaching the far end is at most three pointer hops, not two hundred. Compare the two extremes. The node-per-item list would need 200 separate allocations and 199 links to cross. A single array would hold all 200 in one slab, but a front insert would shove all 200 references over by one. The deque sits in between by design: 64-wide runs you can stream through, stitched by a handful of links you can grow cheaply at either end.
The numbers back it up (all ran on CPython 3.12). A block is 64 × 8 = 512 bytes of cells plus two 8-byte links, so 528 bytes in all. An empty deque already weighs 760 bytes. That is a ~232-byte header plus one pre-allocated block, because a deque is never truly without a block to write into. Append into it and watch sys.getsizeof climb in flat +528-byte steps, one per new block, while the list next to it grows in fine-grained overallocation steps.
import sys
from collections import deque
print(sys.getsizeof(deque())) # 760 (header + one 528 B block)
d = deque()
prev = sys.getsizeof(d)
for i in range(1, 260):
d.append(i)
s = sys.getsizeof(d)
if s != prev: # print only when a new block is hooked on
print(i, prev, "->", s) # 33 760 -> 1288 ; 97 1288 -> 1816 ; ...
prev = sLine by line: the first print confirms the 760-byte floor. Then we append 259 items and print only on the steps, the moments getsizeof changes. The real output was jumps at n = 33, 97, 161, 225, each adding exactly 528 bytes, which is one block. Why the first jump at 33, not 64? Because a fresh deque drops its first item in the middle of the block. An append-only run fills the right half, about ~32 cells, before it needs a second block. The steady-state cost is about 8.25 bytes per element versus a list's flat 8.00. The block links cost roughly a quarter-byte per item, plus up to one block of slack riding at each end.
So why 64, and not 8 or 1,024? The number balances two costs that pull in opposite directions. Make a block too small and you pay for a fresh link, and a possible allocation, far too often. Make it too large and even a two-item deque drags a full block of empty cells at each end. At 64, the per-item bookkeeping falls to a fraction of a byte while the wasted slack stays small. That sweet spot is the constant BLOCKLEN that CPython bakes in.
append or appendleft has somewhere to land without allocating. The deque pays a higher fixed entry fee to guarantee O(1) at both ends from item one. For a handful of items, a list is lighter; the deque earns its keep at scale and at the front.The deeper cut
_collectionsmodule.c the constant is BLOCKLEN = 64, and a fresh deque starts its two cursors near CENTER = (BLOCKLEN - 1) / 2 = 31 — mid-block, not at an edge. That centering is exactly why an append-only run spills into a second block after roughly 32 items, the +528-byte jump we measured at n = 33. Each block is a struct of PyObject *data[64] plus leftlink and rightlink. To avoid hammering the allocator, CPython keeps a small freelist of up to 16 spent blocks: when an end block empties it's parked on the freelist, and the next block that's needed is grabbed from there instead of a fresh malloc. So the "hook on a new block" step is usually just a pointer relink, not a heap allocation — the amortized reasoning we built in Volume 3, made physical.So an append is cheap. But how cheap, and what does it physically do to those cells? →
03What one append actually does
Let's derive the O(1) instead of asserting it. To append(x), CPython looks at the rightmost block and rightindex, the cursor sitting on the last used cell. If there's a free cell to its right, it writes the reference to x into that cell and bumps rightindex by one. That's it: one pointer write, one integer increment. It doesn't matter whether the deque holds ten items or ten million, so the cost is O(1). Only when rightindex already sits on the block's last cell, every 64th append, does more happen. It grabs a block from the freelist, links it on the right, and resets the cursor to the new block's start. That is still constant work. appendleft is the mirror image on the left. pop and popleft read an end cell, step the cursor inward, and unhook a block once one empties. Nothing between the two ends is ever read or moved.
One detail in that derivation is worth pausing on: the freelist. Asking the operating system for memory is expensive, far pricier than a pointer write, so you don't want to do it on every 64th append. Instead CPython keeps a small stash of recently emptied blocks. When a pop drains a block, that block isn't handed back to the OS. It's parked on the freelist. When the next append needs a fresh block, it takes one off the stash instead of allocating. So a workload that churns near a block boundary, appending and popping around the same length, recycles the same blocks over and over and almost never touches the allocator. That's how the O(1) stays a cheap O(1) in practice, not just on paper.
append is a single reference write into the next free cell plus a cursor bump. Because a block's 64 cells are contiguous, a run of appends streams through the same cache lines — O(1) and cache-friendly.from collections import deque
up_next = deque(["Levels", "One More Time", "Strobe"])
up_next.append("Titanium") # add at the back O(1)
up_next.appendleft("Encore") # cut in at the front O(1)
first = up_next.popleft() # serve the front O(1) -> "Encore"
last = up_next.pop() # drop the back O(1) -> "Titanium"Four operations, and all four constant-time moves — no shifting, no reallocation, no walking. The widget below makes that claim literal, so drive it and watch the two cursors crawl outward as you push and inward as you pop. Keep an eye out for the exact moment a full block fills and hooks a fresh one on.
Both ends are O(1). So is a deque just a better list you should use everywhere? Try to grab the middle element and the answer arrives fast. →
04The catch: the middle is O(n)
Ask a list for a[i] and it computes one address, base + i × 8, and jumps there. That's the payoff of a contiguous block: O(1) random access, the same one-multiply-and-add we proved when we built the array. A deque can't do that, because its cells are scattered across linked blocks with no single base. To reach dq[i] it starts at the nearer end and walks the chain block by block, 64 items per hop, until it lands in the right block and indexes inside. That walk is proportional to how deep i sits, so it is O(n).
len(d) answers, d[0] answers, for x in d walks them exactly as before. So it is my list, just laid out differently inside. Nobody arrives there carelessly. Every observation propping it up is true.from collections import deque
import json
d = deque([1, 2, 3])
print("same items:", d[1] == [1, 2, 3][1], "| same length:", len(d) == 3)
print("d == [1, 2, 3] ->", d == [1, 2, 3])
try:
json.dumps({"ids": d}) # the boundary where data leaves the process
except TypeError as e:
print("json.dumps ->", f"{type(e).__name__}: {e}")
print("hasattr(d, 'sort') ->", hasattr(d, "sort"), "| sorted(d) ->", sorted(d))
print("list(d) == [1, 2, 3] ->", list(d) == [1, 2, 3])same items: True | same length: True d == [1, 2, 3] -> False json.dumps -> TypeError: Object of type deque is not JSON serializable hasattr(d, 'sort') -> False | sorted(d) -> [1, 2, 3] list(d) == [1, 2, 3] -> True
d.__eq__([1, 2, 3]) runs and returns NotImplemented, the reflected [1, 2, 3].__eq__(d) returns NotImplemented too, and when both operands decline Python falls back to identity: two different objects, so False. Nothing is broken. Python is refusing to guess that two different containers holding the same things are the same value, which is the identical reason [1, 2] == (1, 2) is False.Two labelled refinements, so none of this reads as a half-truth. First, the deque is not a stripped-down list. Measured on this build,
isinstance(d, collections.abc.MutableSequence) is True, and insert, index, count, remove, copy, clear and reversed(d) are all present and working. What is absent is one family and one family only — the moves that need a random-access middle. There is no in-place sort (line four; sorted(d) works fine and hands you back a list), and there is no slice syntax, whose refusal the toolbelt further down this section spells out along with what to use instead. Second, a great deal of code genuinely cannot tell the two apart: anything that merely iterates your container never notices a thing. What notices is code that compares it against a list, serialises it, sorts it in place, or concatenates — d + [4] raises TypeError: can only concatenate deque (not "list") to deque.So the habit is the run's last line. Keep the deque where the ends are being worked, and spend one
list(d) at the boundary where the data leaves — a single O(n) copy at the edge of the system, deliberately placed, instead of a hidden one inside the loop.I measured it on a million-item deque. Reaching dq[n//2] took about 47 microseconds, while list[n//2] took about 0.17 microseconds — roughly 280× slower, and that gap only widens as n grows. This is not a small constant you can shrug off; it is a different complexity class altogether. The deque is a specialist for the ends, and paying for the middle is exactly how you misuse it.
It helps to count the hops. To land on dq[500000] in that million-item deque, the walk starts at the nearer end and crosses one block at a time, 64 items per hop. That is about 500000 ÷ 64 ≈ 7,800 hops before it arrives. Each hop follows a next pointer into a block sitting elsewhere in memory, so most of them are a fresh cache miss. Seven-odd thousand misses is where those 47 microseconds live, while a list skips every one of them with a single address computation.
Myth
"A deque is just a faster list — swap it in anywhere you use a list."
Reality
A deque is an ends specialist. The front and back are O(1), but indexing, slicing, or inserting in the middle all cost O(n). So the real question is where your code touches the data. If you keep reaching for element i, you want a list. If you only ever add and remove at the ends, you want a deque.
Here's the trap that catches people, and it hides in a loop. Suppose you want to sum a deque and you write for i in range(len(dq)): total += dq[i]. Each dq[i] walks the block chain from the nearer end, so the loop is O(n) per step and O(n²) overall, and a million-item deque would crawl. The fix costs nothing but a habit: iterate directly with for x in dq. That path walks the blocks once, front to back, streaming through each block's 64 contiguous cells, for O(n) total. Same data, same answer, but one version indexes into the middle a million times and the other never does. When you reach for a deque, reach for its iterator too, not its index.
insert(0, x) shifts every reference along; the deque writes one cell in the front block and steps a cursor. Same idea as append, mirrored.append + pop is a stack (Chapter 51); append + popleft is a queue (Chapter 52). The promise never lives in the deque — it lives in which pair you call.extendleft(["Y", "Z"]) put Z in front of Y. Not a bug — it is what “appendleft, repeatedly” must mean.append drops from the left and an appendleft drops from the right: the far end always loses. Read d.maxlen back to find out whether a deque you were handed is bounded at all.for x in d, which walks each block's 64 contiguous cells once — never for i in range(len(d)), which re-walks the chain on every step.INPUTfrom collections import deque
up_next = deque(["Levels", "Strobe"])
up_next.append("Faded") # BACK - "add to queue"
up_next.appendleft("Encore") # FRONT - "play next"
print(list(up_next))
print("plays now ->", up_next.popleft()) # FRONT out
print("undo add ->", up_next.pop()) # BACK out
print(list(up_next))
up_next.rotate(1) # in place, right by 1 - and returns None
print("rotate(1) ->", list(up_next))
print("rotate returns:", up_next.rotate(-1))
print("rotate(-1) ->", list(up_next))
up_next.extend(["A", "B"]) # bulk, at the back, in order
up_next.extendleft(["Y", "Z"]) # bulk, at the front - ONE AT A TIME, so it flips
print("after extends:", list(up_next))
history = deque(maxlen=3) # a self-trimming window
for page in ["home", "docs", "api", "faq", "blog"]:
history.append(page)
print("last 3 pages:", list(history), "| maxlen =", history.maxlen)
# both ends at once: a palindrome check that never slices
def is_pal(s):
d = deque(s)
while len(d) > 1:
if d.popleft() != d.pop(): # one from each end, O(1) each
return False
return True
print("is_pal('racecar'):", is_pal("racecar"), "| is_pal('deque'):", is_pal("deque"))OUTPUT['Encore', 'Levels', 'Strobe', 'Faded']
plays now -> Encore
undo add -> Faded
['Levels', 'Strobe']
rotate(1) -> ['Strobe', 'Levels']
rotate returns: None
rotate(-1) -> ['Levels', 'Strobe']
after extends: ['Z', 'Y', 'Levels', 'Strobe', 'A', 'B']
last 3 pages: ['api', 'faq', 'blog'] | maxlen = 3
is_pal('racecar'): True | is_pal('deque'): False- The middle is the bill for the ends, and it is a real bill. Measured just now on 100,000 items:
L[n//2]took 11.3 ns whileD[n//2]took 1550.6 ns — 137× — even thoughD[0]stayed at 17.9 ns. Sofor i in range(len(d)): use(d[i])is a hidden O(n²);for x in dis the same walk done once. rotatemutates in place and returns None.d.rotate(2)printedNonewhileditself became[4, 5, 1, 2, 3]; writed2 = d2.rotate(2)and you have thrown the deque away and keptNone. Same trap aslist.sort()— a method that reorders in place hands back nothing on purpose.- You cannot slice a deque.
d[1:3]raisesTypeError: sequence index must be integer, not 'slice'. Useitertools.islice(d, 1, 3), which gave[2, 3]. The refusal is honest rather than a gap: a slice would have to walk the block chain anyway, so Python declines to hide an O(n) walk behind a syntax that looks free.
for i in range(len(dq)): use(dq[i]) is a hidden O(n²) — each dq[i] re-walks the chain. If you must scan a deque, iterate it (for x in dq), which the C code walks block by block in one O(n) pass. And if you find yourself wanting random access at all, you probably wanted a list."Ends-only" sounds like a limitation. It's the opposite — a whole family of features you use daily are pure ends games. Start with the one running in your browser right now. →
05A history that forgets — maxlen
Build a deque with maxlen=N and it becomes a fixed-size window on a stream. Once it holds N items, every append silently drops one off the other end to make room. That gives you a rolling "last N" in one line: newest kept, oldest evicted for free. There's no manual trimming, and crucially no O(n) pop(0).
This is the shape behind a lot of everyday plumbing. A live dashboard that shows the last 100 latency samples is a deque(maxlen=100). Each new measurement appends, the hundred-and-first push quietly drops the oldest, and the window slides forward on its own. A chat client keeping the last 50 messages in memory works the same way. So does a moving average: sum the deque and divide by its length, and because the oldest sample fell off for free, you never pay to trim. Without maxlen you'd write the eviction by hand with pop(0), and that's the exact O(n) front-shift we started the chapter warning against.
from collections import deque
recent = deque(maxlen=3) # keep only the 3 most recent
for page in ["A", "B", "C", "D", "E"]:
recent.append(page)
print(list(recent))
# ['A'] -> ['A','B'] -> ['A','B','C'] -> ['B','C','D'] -> ['C','D','E']I ran it. Once the deque is full at three, appending "D" shoulders out "A", and "E" shoulders out "B". You're always looking at the three newest, and the eviction is an O(1) drop at the far end, not a whole-array shuffle. That single line is the skeleton of a surprising amount of software.
Here is the payoff made concrete. Say you keep the last three sensor readings in a deque(maxlen=3) holding [10, 20, 30], whose mean is 20. A new reading of 40 arrives, you append it, and the deque becomes [20, 30, 40] — the oldest value, 10, fell off the far end for free. Now the mean is 30, and you never paid an O(n) shift to drop the stale sample. That is a moving average that trims itself, and it is why sensor feeds and dashboards reach for a bounded deque.
tail -f), a moving average over the last N samples in a trading or sensor feed — all bounded deques. And in the guts of parallel runtimes, work-stealing schedulers (Go, Java's ForkJoin, Rust's Tokio) give each worker its own deque of tasks: the worker pops from one end while idle threads steal from the other — two ends, two customers, zero contention in the common case. You reach for Back in your browser without a thought; you were using a deque.maxlen uses a deque as a rolling buffer. Now use both ends at once, as deliberate scratch space, and an O(n·k) problem collapses to O(n). →
06The killer app: sliding-window maximum in O(n)
Back in Volume 3 we slid a window across an array and kept its running sum current in O(1), adding the value that entered and subtracting the one that left. Maximum is nastier. When the current maximum slides out of the window, there is no cheap way to recover the next one, and re-scanning the whole window is O(k) — which makes the entire pass O(n·k). The fix is a monotonic deque, and it is the deque's signature move.
Before the mechanism, sit with the one idea that makes it work. Say a new value x enters at index j, and some earlier index p still in the deque holds a value ≤ x. Then p is finished. It lies to the left of j, so every future window that still holds p also holds the newer, at-least-as-big x. The older value can never win as a maximum again, so we discard it the instant x arrives. That one observation is why the deque stays short and the whole pass stays O(n).
Keep a deque of indices whose values are strictly decreasing from front to back. For each new element, three steps: (1) pop indices off the back whose values are ≤ the newcomer — while this bigger, newer value stands, they can never again be a window maximum, so discard them; (2) append the new index at the back; (3) if the front index has slid out of the window, popleft it. After those, the front index always holds the current window's maximum. Every index is appended once and removed once across the whole run, so the total work is O(n) — amortized, exactly the reasoning we built in Volume 3.
Let's watch the invariant hold on a real array. Take a = [1, 3, -1, -3, 5, 3, 6, 7] with a window of k = 3, and track the deque of indices. We push index 0, value 1. At index 1, value 3, the newcomer beats the tail value 1, so we evict index 0 and push 1. The deque now reads [1]. At index 2, value -1, the newcomer doesn't beat 3, so it just joins the back: [1, 2]. The window is full now, and the front value a[1] = 3 is the first maximum. Push to index 4, value 5, after index 3 (value -3) has joined the back. Now 5 beats the tail values -3, -1, and 3 in turn, so every index pops and 5 stands alone at the front. Each index entered once and left once, and the front was always the answer. That's the O(n) you were promised, traced by hand.
from collections import deque
def max_sliding_window(a, k):
dq = deque() # holds INDICES, values decreasing front->back
out = []
for i, x in enumerate(a):
while dq and a[dq[-1]] <= x: # (1) evict smaller tails: they can't be max
dq.pop()
dq.append(i) # (2) newcomer joins the back
if dq[0] <= i - k: # (3) front slid out of the window?
dq.popleft()
if i >= k - 1:
out.append(a[dq[0]]) # front index = this window's maximum
return out
print(max_sliding_window([1, 3, -1, -3, 5, 3, 6, 7], 3)) # [3, 3, 5, 5, 6, 7]Walk it: dq never holds values, only indices — that's how it also knows when a candidate expires. The while loop (1) clears every tail index whose value the newcomer x beats or ties, keeping the deque decreasing. Step (2) appends the newcomer at the back. Step (3) checks the front: if its index is ≤ i - k it has fallen off the window's left edge, so popleft. From i = k-1 on, a[dq[0]] — the value at the front index — is the window maximum, emitted in O(1). I ran it against a brute-force max(a[i:i+k]): identical output [3, 3, 5, 5, 6, 7]. On a bigger case (n = 200,000, k = 1000) the deque version took about 80 ms versus the brute force's ~4.8 s — roughly 59× faster here (the gap grows with k; wall-clock varies by machine, the O(n) vs O(n·k) shape does not).
That is the deque's reason to exist, generalized: it is O(1) scratch space with two working ends. A palindrome check pops a character off each end and compares the pair. Balanced-parenthesis checks and BFS frontiers — the breadth-first search we walked through in Volume 3 — lean on the same two-ended access. So does rotate(k), which quietly powers round-robin scheduling. Run deque([1,2,3,4,5]).rotate(2) and you get [4,5,1,2,3] in O(k), no reshuffle.
Why is that rotate O(k) rather than O(n)? Rotating right by k only has to move k references from one end to the other — pop from the back, appendleft onto the front, k times over. The other n − k items never move. The two cursors do all the walking, which is why round-robin scheduling can advance a whole rota by a step and touch only the few items at the edges.
sum(1 for t in hits if t > now - 300), and you are done in one line. It is correct on the first attempt and it stays correct forever, which is exactly why it earns a second job further down as the referee. Then you sharpen it, and the sharpening is genuinely good work rather than a straw man. The timestamps arrive in order, so the list is already sorted, so there is no reason to look at all of them: bisect_right(hits, now - 300) finds the boundary in O(log n) and the answer is everything to its right. Measured in the run below on 200,000 events, 200 queries cost 950,184 us by scanning and 47 us by bisecting — a roughly twenty-thousand-fold win bought with one import. It feels finished.hits still holds every timestamp that ever arrived, so a five-minute question is being answered by a structure that remembers five months. The run prices it — the list ends at 200,000 entries and 6,273.5 KB, which is 32 bytes per event, kept forever, and that figure extends by plain arithmetic: at this stream's 27.8 events a second a full day is about 2.4 million events, so roughly 73 MB a day added to answer a question about the last five minutes. Then you reach for the obvious repair, and the repair is the cost section 01 already priced, met again from the other direction. Same algorithm, one container swapped: expiring the stale front on every arrival took 34.7 ms with popleft and 1,893.1 ms with pop(0) — 55× — and both left the identical 8,233 events live. The instinct was never wrong. It is unaffordable, and it is unaffordable at the one end a list cannot cheaply touch.popleft. Say the rule in those words and the container names itself: arrivals at the back, expiry at the front, both ends busy, the middle never touched once. The accounting arrives with it — every event is appended once and popped once, so the while loop that looks like an inner loop is amortised O(1) per event, and the run counts it out loud at 200,000 appends against 191,767 poplefts.One correction before the code, because it is the piece people get backwards here: maxlen does not solve this problem.
maxlen bounds how many, and this window is bounded by how old — the live count wandered between roughly 8,200 and 8,555 as traffic breathed, and no fixed number is the window. The two compose rather than compete. Front-expiry keeps the answer correct; a generous maxlen is a memory fuse for a traffic spike. Set that fuse below the live window and it quietly drops events that are still inside it, which the run's last line shows costing 8,349 live events down to 1,000 — a wrong answer with no error attached, the worst kind.from collections import deque
import random, sys, time, bisect
WINDOW = 300.0 # "how many in the last five minutes?"
class HitWindow:
"""Events inside the last `window` seconds. Arrivals are non-decreasing."""
def __init__(self, window=WINDOW, cap=None):
self.window, self._w = window, deque(maxlen=cap) # cap guards MEMORY, not age
def record(self, t):
self._w.append(t) # newest joins the back, O(1)
def count(self, now):
cutoff = now - self.window
while self._w and self._w[0] <= cutoff:
self._w.popleft() # the oldest is ALWAYS at the front
return len(self._w)
def by_scan(hist, now): return sum(1 for t in hist if t > now - WINDOW) # the instinct
def by_bisect(hist, now): return len(hist) - bisect.bisect_right(hist, now - WINDOW)
random.seed(11)
N, SPAN = 200_000, 7200.0 # 200k events across two hours
stream = sorted(random.uniform(0.0, SPAN) for _ in range(N))
hw, hist, peak, ok = HitWindow(), [], 0, True
for i, t in enumerate(stream):
hw.record(t); hist.append(t)
if i % 1000 == 999: # a query every 1000 events
live = hw.count(t)
if not (live == by_scan(hist, t) == by_bisect(hist, t)): ok = False
peak = max(peak, live)
kb = lambda c: (sys.getsizeof(c) + 24 * len(c)) / 1024
print(f"{N:,} events over {SPAN:,.0f}s (avg {N/SPAN:.1f}/s) | window {WINDOW:.0f}s")
print(f"all three agree on all {N//1000} queries: {ok}")
print(f"deque holds {peak:>8,} at peak history list holds {len(hist):>8,}")
print(f"appends {N:,} / poplefts {N - len(hw._w):,} -> each event enters once, leaves once")
print(f"memory deque + floats {kb(hw._w):>8,.1f} KB list + floats {kb(hist):>9,.1f} KB"
f" ({kb(hist)*1024/N:.0f} B/event, kept forever)")
def ms(fn, xs):
t0 = time.perf_counter()
for x in xs: fn(x)
return (time.perf_counter() - t0) * 1e6
q = stream[-200:]
print(f"200 counts deque front-expiry {ms(hw.count, q):>10,.0f} us"
f" | bisect {ms(lambda t: by_bisect(hist, t), q):>7,.0f} us"
f" | full scan {ms(lambda t: by_scan(hist, t), q):>10,.0f} us")
t0 = time.perf_counter() # same algorithm, both containers
dq = deque()
for t in stream:
dq.append(t)
while dq and dq[0] <= t - WINDOW: dq.popleft()
d_ms = (time.perf_counter() - t0) * 1000
t0 = time.perf_counter()
li = []
for t in stream:
li.append(t)
while li and li[0] <= t - WINDOW: li.pop(0)
l_ms = (time.perf_counter() - t0) * 1000
print(f"expiring on EVERY arrival: popleft {d_ms:,.1f} ms vs pop(0) {l_ms:,.1f} ms"
f" ({l_ms/d_ms:,.0f}x, both left {len(li):,} live)")
loose, tight = HitWindow(), HitWindow(cap=1000)
for t in stream[:20_000]: loose.record(t); tight.record(t)
print(f"cap=None -> {loose.count(stream[19_999]):,} live | cap=1000 -> {tight.count(stream[19_999]):,} live"
" <- too small a cap drops LIVE events")200,000 events over 7,200s (avg 27.8/s) | window 300s all three agree on all 200 queries: True deque holds 8,555 at peak history list holds 200,000 appends 200,000 / poplefts 191,767 -> each event enters once, leaves once memory deque + floats 260.2 KB list + floats 6,273.5 KB (32 B/event, kept forever) 200 counts deque front-expiry 25 us | bisect 47 us | full scan 950,184 us expiring on EVERY arrival: popleft 34.7 ms vs pop(0) 1,893.1 ms (55x, both left 8,233 live) cap=None -> 8,349 live | cap=1000 -> 1,000 live <- too small a cap drops LIVE events
popleft wearing a disguise. Look for that shape in the specification rather than in the data: “count the last N seconds”, “keep the last ten minutes of an audit trail in memory”, “ignore a message you have already seen inside a five-minute window”, “drop anything older than T” — every one of them retires its oldest arrival first, and every one of them is quietly quadratic if you build it on a list. Then learn the near-miss, because a tell that fires on everything is not a tell. An idle timeout — a session that dies thirty minutes after its last request rather than thirty minutes after login — breaks the rule outright: refresh one session and its expiry jumps ahead of sessions that arrived later, so the front is no longer guaranteed to be the next to go, and this whole design quietly stops being correct. (Fixed lifetime from login, and it is back in the family.) The second habit here is worth as much as the first: keep the naive version and let it referee. by_scan is the line you outgrew, and that is exactly what makes it a good oracle — it was written before you were clever, so it cannot share your clever mistake. Run it beside the fast one on every query, retire it only once it has agreed a few hundred times in a row, and when you quote the numbers afterwards, quote the shape alongside them: wall-clock varies by machine and by run, but O(1) expiry against unbounded memory does not.RollingMean(n): it takes readings one at a time and reports the mean of the last n. Storage is a deque(maxlen=n) and nothing else — no counters, no manual trimming, no pop(0).Then keep yourself honest. Write a second, dumber version that keeps every reading in a plain list and re-slices the last n each time. Run both over the same readings and assert the two lists of means are identical — not close, identical. A rolling window that quietly disagrees with the obvious calculation is worse than no window at all.
Then the twist, and it is the real lesson. Summing the window on every call is O(n). The clever fix is obvious: keep a running total, subtract the value about to fall off, add the newcomer — O(1) forever. Build that too. Run 200,000 random readings through both, compare the final answers with
==, and report exactly what you find. Do not guess. Print the two numbers with repr and look at every digit.show the solution
from collections import deque
import random
class RollingMean:
"""Mean of the last N readings. The window trims itself."""
def __init__(self, n):
self._w = deque(maxlen=n) # the ENTIRE eviction policy
def add(self, x):
self._w.append(x) # the (n+1)th append drops the oldest, O(1)
return self.mean
@property
def mean(self):
return sum(self._w) / len(self._w) if self._w else None
def __len__(self):
return len(self._w)
def naive_means(xs, n):
"""The version with no deque - kept honest by slicing a plain list."""
seen, out = [], []
for x in xs:
seen.append(x)
w = seen[-n:] # the last n, recopied every single time
out.append(sum(w) / len(w))
return out
readings = [12.5, 13.0, 11.75, 14.25, 13.5, 12.0, 15.5, 11.0]
r = RollingMean(3)
mine = [r.add(x) for x in readings]
theirs = naive_means(readings, 3)
for x, a, b in zip(readings, mine, theirs):
print(f" add {x:6} -> rolling {a:.6f} naive {b:.6f} {'ok' if a == b else 'DIFFER'}")
print("identical:", mine == theirs)
print("window after 8 readings:", list(r._w), "| len", len(r))
# add 12.5 -> rolling 12.500000 naive 12.500000 ok
# add 13.0 -> rolling 12.750000 naive 12.750000 ok
# add 11.75 -> rolling 12.416667 naive 12.416667 ok
# add 14.25 -> rolling 13.000000 naive 13.000000 ok
# add 13.5 -> rolling 13.166667 naive 13.166667 ok
# add 12.0 -> rolling 13.250000 naive 13.250000 ok
# add 15.5 -> rolling 13.666667 naive 13.666667 ok
# add 11.0 -> rolling 12.833333 naive 12.833333 ok
# identical: True
# window after 8 readings: [12.0, 15.5, 11.0] | len 3
# THE TWIST - the "clever" O(1) incremental version.
class DriftingMean:
def __init__(self, n):
self._w, self._total = deque(maxlen=n), 0.0
def add(self, x):
if len(self._w) == self._w.maxlen:
self._total -= self._w[0] # subtract the one about to fall off
self._w.append(x)
self._total += x
return self._total / len(self._w)
random.seed(7)
stream = [random.random() * 1e6 for _ in range(200_000)]
d, e = DriftingMean(50), RollingMean(50)
for x in stream:
a, b = d.add(x), e.add(x)
print(f"after 200,000 readings: incremental {a!r}")
print(f" re-summed {b!r}")
print("equal:", a == b, "| absolute gap:", abs(a - b))
# after 200,000 readings: incremental 474253.99073524325
# re-summed 474253.9907352406
# equal: False | absolute gap: 2.6775524020195007e-09
# WHAT JUST HAPPENED. The deque half is exact: maxlen dropped the right
# value at the right moment, 200,000 times, with no bookkeeping from us.
# The arithmetic half is not. Every float add and subtract rounds to the
# nearest representable number, and the running total carries all 200,000
# of those roundings forward forever. The re-summed version throws its
# total away and rebuilds it from the 50 values actually in the window,
# so it can only ever be 50 roundings deep. After 200,000 readings the two
# have drifted apart by ~2.7e-09 - tiny, real, and growing.
#
# So which do you ship? Neither, blindly:
# - re-summing is O(n) per reading but its error never accumulates. For
# a window of 50 that is 50 adds - nothing. Default to this.
# - the incremental total is O(1) and pays for it in drift. Reach for it
# only when n is large enough that O(n) hurts, and then either use
# math.fsum for the periodic rebuild or re-sum every few thousand
# readings to reset the error.
# The deque was never the risky part. The float was.i, that "yes" turns to "no" — and you want a list.Every linear structure so far — array, stack, queue, deque — reaches its data by walking: to a position, or to an end. The next family throws walking out. Hand it a key — a title, a name, a label — and it computes the address and lands on the value in a single hop, no matter how many million entries sit inside. How can a key become an address? →
- A deque is a stack and a queue fused into one object —
append,appendleft,popandpopleftare every one of them O(1) — and that is the offer a list cannot match, because a list's front costs a shift of every remaining reference, which is why draining 50,000 items withpop(0)took about 4.7 seconds against the deque's 4 milliseconds. - The trick that buys it is linking the blocks instead of the items: 64 references sit shoulder to shoulder inside one 528-byte block and only the blocks are chained, so you get a linked list's cheap ends while keeping most of an array's cache locality — and one
appendis a single reference write into a free cell plus a cursor bump, with a fresh block hooked on only when an end block fills. - The bill for two fast ends is a hostile middle:
d[0]andd[-1]stay O(1) forever while anything between them walks the block chain 64 items at a time, so scan withfor x in dand never with an index inside a loop — the same walk done once instead of O(n²). maxlenturns a deque into a window that trims itself: once it is full the far end always loses, so a rolling “last N” costs one cursor step per arrival rather than a whole-array shuffle — and it bounds how many, never how old, which is why a window measured in seconds still needs an expiry loop at the front.- The signature move is a deque of indices kept decreasing from front to back: a newcomer evicts every smaller tail at the back because a bigger, newer value shadows them forever, the front retires when it ages out of the window, and the front index then names that window's maximum for free — each index in once and out once, so O(n·k) collapses to O(n).
A deque refuses to choose between the front and the back — it makes both ends equally cheap — so these twelve tiny programs let you feel that symmetry first, then watch it earn its keep as a self-trimming window and, finally, in the deque's signature sliding-window-maximum trick.