11Memory, mastered

In Chapter 10 we learned to stream — to let a generator hand us one item at a time, so we never had to hold the whole thing at once. This chapter turns to what we do hold. It asks the one question a profiler eventually forces on you: how many bytes did that object actually cost? Here's the plan. First we weigh things honestly. Python gives you a single number, and there's a far larger truth hiding behind it. Then we start cutting. We pack raw numbers wall-to-wall with array, delete a hidden table off every object with __slots__, hand a cache the power to let go with weakref, spend a little memory to buy speed with lru_cache, and finally page a file bigger than RAM through a sliver of it with memmap. By the end, we've taken Python's flexible-but-heavy defaults apart and bought back every byte we weren't using. No fatter machine, no less data — just knowing exactly what we're storing and what it costs.

01First, weigh it

Chapter 3 in one breath: every object carries a reference count, and the instant that count hits zero the object is freed. Behind it runs a generational collector, sweeping up the reference cycles the counter alone can't see. All that machinery answers exactly one question: when does memory come back? Now we turn the other way. The question you'll actually feel in a profiler isn't when — it's how much did you hand over in the first place. Here's the catch: you cannot shrink a footprint you cannot read. So we start at the scale.

The scale is a single function — sys.getsizeof(obj). Point it at an object and it reports the bytes that object occupies. That's it. Point it at an integer and you get 28: the same 28-byte parcel you met back in chapter 2, a plain machine number wrapped in Python's object bookkeeping. Point it at a list of ten track lengths and you get 136. Two clean numbers. And almost everyone reads that second one wrong.

Read that 136 slowly, because it decomposes exactly: 136 = 56 bytes of list header + 10 pointers × 8 bytes. That is the container only. The ten pointers are just addresses — each one a signpost reading "the real number is over there." Over there sit ten separate int objects, 28 bytes apiece. That's 280 more bytes scattered across the heap, and getsizeof never mentioned them. The list didn't store your numbers. It stored directions to them.

import sys

# ten track lengths in milliseconds — exactly what a music API hands you
# (Blinding Lights = 200000, Titanium = 245000, then eight more)
lengths_ms = [200000, 245000, 238000, 194000, 260000,
              215000, 222000, 301000, 244000, 313000]

print(sys.getsizeof(200000))      # 28  — one boxed int, weighed alone
print(sys.getsizeof(lengths_ms))  # 136 — the list CONTAINER only

So ten numbers cost 416 bytes to store 40 bytes of actual number — a 90% tax, almost all of it pointers and per-int object headers. What if we threw the boxes away and packed the raw numbers wall-to-wall, four bytes each? That's the trade an array makes against a list — next. →

02Pack the numbers: array vs list

Back in chapter 6 you paid for a list's flexibility. Now you learn the receipt. Every slot in a list is a pointer — an 8-byte address that says only "the real thing lives over there." Store a playlist's track lengths in a list and each number costs you twice. There's the 8-byte slot to point at it, plus a fully boxed int object sitting elsewhere in RAM — header, refcount and all — weighing 28 bytes. That is 36 bytes to remember one number a pocket calculator holds in four. A thousand songs, and you've spent 36 KB pointing at numbers that would fit in 4.

The array module — already in the standard library, no install — makes the opposite bargain. Its whole promise is one word: homogeneous. Tell it up front that every element is the same kind of number, and it throws the boxes away entirely. array('i', …) stores raw machine integers — the bare 4-byte values the processor adds and compares directly, with no Python object wrapped around them — laid shoulder to shoulder in one unbroken block. That little 'i' is a typecode. It means "signed 4-byte int," and it fixes the width of every cell at itemsize = 4 bytes, for the life of the array. Same numbers, a ninth of the memory.

Watch Python confirm the shape. A real music library — Spotify's API is one — keeps every duration in milliseconds. So "Blinding Lights" is 200000 and "Titanium" is 245000: numbers far too big to squeeze into a single byte, and each one genuinely its own object once it lands in a list. Pack them into an array instead and the whole thing becomes one flat ribbon of bytes. This is the same trick one scale up, and it's what makes audio possible at all. A waveform is nothing but a long run of integer samples — roughly 44,100 numbers for every second of sound — and no machine could hold a three-minute track as millions of separately boxed int objects. Packed wall-to-wall, a whole song is one clean block of bytes. In section 6 you'll page exactly such a block off disk without ever loading it whole.

from array import array
import sys

# Spotify hands you every duration in milliseconds
lengths = array('i', [200000, 245000, 238000, 194000, 260000])
#                     ↑ Blinding Lights   ↑ Titanium

print(lengths.itemsize)        # 4   — bytes per cell, fixed for life
print(sys.getsizeof(lengths))  # 100 — header 80 + 5×4, the WHOLE bill
print(lengths[2])              # 238000 — still indexes like a list

lengths.append('quiet')          # TypeError: 'str' object cannot be
#                                    interpreted as an integer

Two things just happened that never happen with a list. First, sys.getsizeof told the whole truth — 100 bytes, nothing hiding — because there are no boxes stashed elsewhere to chase down and add up. (Ask a list its size and you get only its slot array. The 28-byte boxes it points at are not counted, and totalling them is on you.) Second, the flexibility is gone by design. An array('i') holds ints and only ints. Try to append a string and Python stops you cold with a TypeError — the cell is four raw bytes, with nowhere to put a word.

Arrays pack numbers. But a Song is not a number — it has a title, an artist, a length, each with a name. How do you shrink an object whose fields you look up by name? →

03Shrink every object: __slots__

Every section so far has watched Python trade memory for freedom, and every time the freedom turned out heavier than it looked. Here is the heaviest habit of all, and the one line that breaks it. You haven't built a class yet — that's chapter 12 — so here's just enough to stand on. A class is a stamp, an instance is one object stamped from it, and writing s.year = 2019 sets an attribute on that instance. Where does Python keep those attributes? In a per-instance __dict__ — a full hash table (chapter 7) mapping each attribute name to its value, one private table hanging off every object. That little dictionary is exactly what lets you bolt a new attribute onto any object, any time. It's also why one plain Song costs, on CPython 3.12 (64-bit), a 48-byte object plus the 296-byte dict it points at — 344 bytes of skeleton per song, before the title string has spent a single byte.

__slots__ is the deal offered in return. Declare the complete list of attribute names up front and Python does something drastic. It deletes the per-instance dict entirely and reserves a fixed position — a slot — for each name, right inside the object. The same three values, but now they live in the instance instead of in a heavy table hanging off it. The 296-byte drawer simply never gets built.

import sys

class PlainSong:                       # an ordinary class — chapter 12 builds these in full
    def __init__(self, title, artist, seconds):
        self.title, self.artist, self.seconds = title, artist, seconds

class Song:
    __slots__ = ('title', 'artist', 'seconds')   # the COMPLETE attribute list, up front
    def __init__(self, title, artist, seconds):
        self.title, self.artist, self.seconds = title, artist, seconds

p = PlainSong('Blinding Lights', 'The Weeknd', 200)
s = Song('Blinding Lights', 'The Weeknd', 200)

print(sys.getsizeof(p) + sys.getsizeof(p.__dict__))   # 344  = 48 + 296
print(sys.getsizeof(s))                               # 56   — and no __dict__ exists at all
print(hasattr(s, '__dict__'))                         # False

Run it and Python states the whole case in three numbers: 344, 56, False. Notice what the last line means: the dict didn't shrink, it vanished. A slotted Song has no __dict__ to ask about.

One object saving 288 bytes is a rounding error you'd never feel. The trade only pays off at scale. Picture a music service holding its whole catalogue in memory at once. Slide the library size, then flip the switch and watch the total fall off a cliff.

You can shrink an object right down to the bone — and it still costs those bytes forever if something refuses to let it die. The usual culprit is the very thing built to help: a cache clutching a reference long after everyone else has moved on. What if a reference could point at an object and still, when the time came, simply let go? →

04The reference that lets go: weakref

Back in chapter 3 you learned the one rule that governs every object's life. It lives for exactly as long as its reference count stays above zero, and the instant the count hits zero the bytes are handed back. That rule is a gift: automatic cleanup, no free() to forget. But it has a shadow. Any long-lived container that stores an object is itself a reference, so it keeps that object's count above zero for as long as the container lives. Build an album-art cache as a plain dict, drop "Blinding Lights" from the playlist, and the song's bytes still won't come back — the cache is still holding the count up. Nobody wants the object any more, yet it cannot die. That's a memory leak with the best of intentions.

A weak reference is the way out. It points at an object without counting — it adds 0 to the reference count. The object's life is decided by the strong references alone; the weak one just watches from the side of the road. weakref.WeakValueDictionary is a dict whose values are held this way. The moment the last strong reference to a value dies, that entry doesn't linger — it removes itself. The cache stops being a reason to stay alive and goes back to being what a cache should be: a convenience that never outstays its welcome.

In code the whole trick is one import and one dictionary. Watch the length of the cache drop to zero on its own, the instant the strong reference goes — no del art["Blinding Lights"], no cleanup pass, nothing. And it isn't that the entry polls for a dead value. A WeakValueDictionary stores each value as a weak reference carrying a callback. When the Song's strong count hits zero (chapter 3's one rule) and the object is freed, the interpreter fires that callback, which deletes the key. The same refcount-reaches-zero event that frees the object is what erases its entry — one mechanism, not two.

import weakref

class Song:
    def __init__(self, title):
        self.title = title

art = weakref.WeakValueDictionary()
bl  = Song("Blinding Lights")
art["Blinding Lights"] = bl     # weak — adds 0 to the refcount

print(len(art))    # 1
del bl              # the last STRONG reference is gone → freed at once
print(len(art))    # 0 — the entry removed itself

The raw form makes the answer plain. weakref.ref(bl) is a callable watcher. Call it and it hands you the object while the object lives, and hands you None the moment it's gone. There is no dangling pointer to trip over — the watcher knows its target has died and simply reports the honest truth.

t = Song("Titanium")
r = weakref.ref(t)      # a small, fixed-size watcher — ~80 bytes, whatever t's size
r()                     # → the Song, while it lives
del t                   # last strong ref gone
r()                     # → None — the watcher tells you it's gone

The WeakValueDictionary just wraps that behaviour in dict clothing. After the value dies, the key quietly ceases to exist, and art.get("Blinding Lights", "MISS") returns "MISS" — exactly as if you'd never stored it. The absence is the feature.

weakref stops the accidental hoard — memory you meant to release. But sometimes hoarding is the entire plan: keep the answers you paid for, and hand them back for free. Spend bytes on purpose, buy back time →

05Spend bytes, buy speed: lru_cache

Every trick in this chapter so far has spent one resource to save the other — packing, trimming, and evicting all spend a little time to save memory. This section flips the trade clean around: spend memory to buy speed. The setup is a function that gets asked the same question over and over. Being a diligent machine, it works out the same answer from scratch every single time. Turning 200000 ms into the label "3:20" is cheap. But picture the body doing real work — a database lookup, a network round-trip, a slow calculation — with the same input handed to it a thousand times, and paid for in full on every call. The deeper witness makes it concrete. That same “Blinding Lights” is not just its 200-second duration but 8,820,000 signed-16-bit samples on disk, and taking the FFT energy of one fixed window of them is exactly that kind of expensive work — asked for again, unchanged, every time a spectrogram redraws.

Memoization is the fix, and it's almost embarrassingly simple. Keep a note of every answer you have already worked out, filed under the arguments that produced it, and glance at the note before doing the work again. functools.lru_cache bolts exactly that onto any function with a single line. That @lru_cache(...) line is a decorator, and decorators are pure chapter 9: a function is just an object you can hand around. Strip the @ sugar and the line means one plain assignment — as_clock = lru_cache(maxsize=128)(as_clock). Call lru_cache, hand it your function, and rebind the name to the wrapped version it returns. That wrapper checks a hidden dict of past answers first, and only calls your original code on the way to filling that dict.

from functools import lru_cache

@lru_cache(maxsize=128)          # keep the 128 most-recent answers
def as_clock(ms):
    seconds = ms // 1000                          # 200000 ms → 200 s
    return f"{seconds // 60}:{seconds % 60:02d}"  # 185 s → '3:05', never '3:5'

as_clock(200000)             # miss: body runs, answer filed under 200000
as_clock(200000)             # hit:  served from the cache, body never runs
print(as_clock.cache_info())
# CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)

Read that last line like an instrument panel. The first call was a miss (misses=1): no note existed, so the body ran and the answer '3:20' was stored. The second call was a hit (hits=1): the arguments matched a stored key, so the answer came straight back and the body stayed cold. currsize=1 because both calls share one entry, and maxsize=128 is the ceiling we set. Call it three times, ten times, a thousand times with 200000 — the body runs exactly once, and every call after is a free hit.

First, why cap it at all? Leave the cache unbounded (maxsize=None) and a memoized hot function turns into a slow leak. Every never-before-seen argument files one more entry that is never removed, so on an endless stream of distinct inputs the cache quietly swallows your RAM. The bound is the cure, and it forces the question the two letters in the name answer. LRU is least recently used: when the 128 slots are full and a new answer needs a home, which old answer gets thrown out? The rule is to evict the entry that has gone untouched the longest — the least recently used one. The bet is that an answer you keep asking for is worth keeping, while one nobody has wanted in ages can be recomputed if it's ever missed again. Every hit quietly promotes its entry back to "most recent," so popular answers survive and stragglers fall off the end. The memory bill never climbs past maxsize entries. Step through it:

On as_clock the win is real but modest, because the body was cheap to begin with. This was the clearest place to see the mechanism, not to feel the payoff. Point the same one line at a function that genuinely hurts and the trade turns spectacular. Naïve recursive Fibonacci is the classic: fib(32) re-derives the same sub-problems about seven million times. Memoize it and each n is computed exactly once:

from functools import lru_cache

@lru_cache(maxsize=None)         # None = unbounded: never evict
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

fib(32)                          # fills the cache in ~0.02 ms
fib(32)                          # a pure hit: ~0.0003 ms
print(fib.cache_info())
# CacheInfo(hits=31, misses=33, maxsize=None, currsize=33)

Without the cache, fib(32) grinds for a few hundred milliseconds. With it, the whole thing fills in a fraction of a millisecond, and every repeat is a hit measured in ten-thousandths — thousands of times faster, bought with 33 stored integers. Notice the mechanism honestly, though. The cache here isn't only remembering the final answer, it's remembering every sub-problem along the way, and that's what collapses exponential recomputation into a single linear sweep. currsize=33 is precisely the n values 0 through 32, each computed once.

Packing, trimming, evicting, caching — every game in this chapter has been played inside RAM, on data that fits. But some data is simply bigger than every box on the street, and every byte you cache is a byte someone eventually has to clean up. What happens the day the data won't fit at all — and who frees the memory you're done with, on time? →

06Bigger than RAM, cleaned up on time

Here is a size that should scare you a little. The numbers on the slider for "Blinding Lights" — the length, the title — are a handful of bytes. But the song itself, the actual waveform, is 200 s × 44,100 samples per second = 8,820,000 samples. Store each sample as a float64 (8 bytes of precision) and that's 70.56 MB for one track. "Titanium" at 245 s is 86.4 MB. Load a twenty-song playlist the obvious way and you've asked for well over a gigabyte of RAM — for audio you're going to stream past your ears a few thousand samples at a time and never need all at once. The naive move drowns the machine.

numpy.memmap is the escape hatch, and it's a genuinely surprising one: it hands you an array that lives on disk and pretends to be in memory. Here is what actually happens, and it is not a copy. Opening a memmap maps the file's bytes into your process's address space. The array's indices become addresses that translate straight to positions in the file, and at open time nothing is loaded. Only when you touch an address that isn't yet resident does the CPU trip a page fault, and the operating system reads in exactly the one 4,096-byte page that holds it — the smallest parcel the OS will ever move between disk and memory. Read wave[0] and the OS faults in that 4 KB slab; the other 70.55 MB stays on the platter, untouched, costing you nothing. The mapping is address translation, not a load.

import numpy as np                     # pip install numpy

wave = np.memmap("levels.dat",         # 70.56 MB on disk, ~0 in RAM
                 dtype="float64",
                 mode="w+",
                 shape=(8_820_000,))    # 200 s × 44,100 per second
wave[0:3] = [0.0, 0.12, 0.25]          # touching index 0 pages in bytes 0–4095
wave.flush()                           # push buffered writes down to the file
del wave                               # release the mapping when you're done

Drag the reader below and watch the cost appear. The slider says how many seconds of the waveform you scan; the green bar is what that forces the OS to copy into RAM. Notice two things. The file on disk never grows, and RAM fills in page-sized steps — ask for a single sample and you still pay for a whole 4 KB page, because 4 KB is the smallest thing the OS knows how to move.

But look again at the code and you'll spot two chores that arrays don't usually demand: flush(), then del. A plain list never asks that of you. A memmap does, because it is holding something borrowed — an open file, a live mapping into the kernel. Files, mappings, sockets, database connections: these are resources, on loan from the operating system, and the loan sits outside the garbage collector's tidy world of Python objects. The GC frees memory, but it does not reliably flush your buffered bytes to disk. And here is the trap: if your program raises an exception on the line between acquiring the resource and releasing it, the cleanup line never runs at all.

Python's answer is the context manager — any object with two methods. __enter__ runs the moment a with block opens, and __exit__ runs when the block ends no matter how it ends. Normal fall-through, an early return, a raised exception — every path out of the block passes through __exit__. That is the whole guarantee, and it is what makes cleanup un-skippable.

with open("playlist.txt", "w") as f:      # __enter__ → f is the open file
    f.write("Blinding Lights — The Weeknd — 200\n")
    f.write("Titanium — David Guetta ft. Sia — 245\n")
# leaving the block ran __exit__ → f is GUARANTEED closed,
# even if a write() in the middle had raised an error

Every trick in this chapter has been quietly leaning on a word we haven't earned yet: class. Slots on a class, weak references to instances, __enter__ and __exit__ methods on a blueprint. It's time to build the real thing — a Song, a Playlist, and objects from the metal up →