Dictionaries and Collections

Dict merge with | (3.9)

Merging two dicts had three common approaches before 3.9. None of them were obvious at a glance.

Python

1defaults = {"timeout": 30, "retries": 3, "verbose": False}
2overrides = {"timeout": 60, "debug": True}
3
4# Option 1: dict() with unpacking -- verbose
5merged = {**defaults, **overrides}
6
7# Option 2: update() -- mutates in place, no new dict
8config = dict(defaults)
9config.update(overrides)
10
11# Option 3: ChainMap -- lazy, reads from first dict that has the key
12from collections import ChainMap
13merged = dict(ChainMap(overrides, defaults))

Python

1defaults = {"timeout": 30, "retries": 3, "verbose": False}
2overrides = {"timeout": 60, "debug": True}
3
4# Python 3.9: | creates a new merged dict, right side wins on conflicts
5config = defaults | overrides
6print(config)
7
8# |= merges in place, like dict.update()
9config = dict(defaults)
10config |= overrides
11print(config)
12
13# Left-to-right precedence: right operand wins
14print({"a": 1} | {"a": 2})   # {'a': 2}

The practical use: layering configs. Start with defaults, overlay environment-specific settings, overlay user-provided overrides. Each layer is a plain dict merge with |.

Python

1base = {"host": "localhost", "port": 5432, "timeout": 30}
2env  = {"host": "db.prod.example.com", "port": 5432}
3user = {"timeout": 60}
4
5final = base | env | user
6print(final)

Counter arithmetic and most_common

collections.Counter is a dict subclass for counting. It has been in Python since 2.7, but most people use it as a plain tally and miss its arithmetic operators.

Python

1from collections import Counter
2
3words = "the quick brown fox jumps over the lazy dog the fox".split()
4c = Counter(words)
5
6# Frequency table in one line
7print(c.most_common(3))   # top 3
8
9# Add two counters
10morning = Counter(["coffee", "tea", "coffee", "juice"])
11afternoon = Counter(["coffee", "water", "tea", "coffee"])
12total = morning + afternoon
13print(total.most_common())

Python

1from collections import Counter
2
3inventory = Counter({"apple": 10, "banana": 5, "cherry": 3})
4sold      = Counter({"apple": 4, "banana": 8, "cherry": 1})
5
6# Subtraction: removes negative/zero counts
7remaining = inventory - sold
8print(dict(remaining))   # banana drops out (5 - 8 = -3, removed)
9
10# intersection: minimum of each count
11shared = inventory & sold
12print(dict(shared))
13
14# union: maximum of each count
15combined = inventory | sold
16print(dict(combined))

itertools.batched (3.12) and itertools.pairwise (3.10)

batched: fixed-size chunks without the slice gymnastics

Splitting a list into fixed-size chunks was a famous Python interview snippet because there was no built-in for it. Now there is.

Python

1# Before 3.12: the classic chunk recipe
2def chunked(iterable, n):
3    lst = list(iterable)
4    return [lst[i:i + n] for i in range(0, len(lst), n)]
5
6items = range(10)
7for batch in chunked(items, 3):
8    print(batch)

Python

1from itertools import batched
2
3# Python 3.12: lazy, no need to materialise the list first
4items = range(10)
5for batch in batched(items, 3):
6    print(batch)   # each batch is a tuple
7
8# Practical: insert rows in batches of 1000
9def insert_rows(rows: list[dict]) -> None:
10    for batch in batched(rows, 1000):
11        db.bulk_insert(batch)

pairwise: consecutive overlapping pairs

Python

1from itertools import pairwise
2
3# Before 3.10: zip with offset slice
4points = [0, 10, 25, 30, 50]
5deltas_old = [b - a for a, b in zip(points, points[1:])]
6
7# Python 3.10: pairwise -- cleaner, lazy
8deltas = [b - a for a, b in pairwise(points)]
9print(deltas)   # [10, 15, 5, 20]
10
11# Useful for: consecutive diffs, sliding window checks, path lengths
12path = ["A", "B", "C", "D"]
13edges = list(pairwise(path))
14print(edges)   # [('A', 'B'), ('B', 'C'), ('C', 'D')]

ChainMap: layered lookup without merging

ChainMap groups multiple dicts into a single view. Lookups search each dict in order and return the first match. No copy is made. Writes always go to the first dict.

Python

1from collections import ChainMap
2
3# Classic use: environment variable layering
4defaults     = {"debug": False, "log_level": "INFO", "timeout": 30}
5env_config   = {"log_level": "WARNING"}               # env overrides some
6user_config  = {"debug": True, "timeout": 60}         # user overrides more
7
8config = ChainMap(user_config, env_config, defaults)
9
10print(config["debug"])      # True   -- from user_config
11print(config["log_level"])  # WARNING -- from env_config
12print(config["timeout"])    # 60     -- from user_config
13print(config["missing"])    # KeyError -- not in any layer

Python

1from collections import ChainMap
2
3# Writes go to the first map only -- originals untouched
4user = {"x": 1}
5base = {"x": 99, "y": 2}
6
7layered = ChainMap(user, base)
8layered["z"] = 100   # writes to 'user', not 'base'
9
10print(dict(user))    # {'x': 1, 'z': 100}
11print(dict(base))    # {'x': 99, 'y': 2}
12
13# new_child() creates a fresh empty layer on top -- great for scopes
14child = layered.new_child({"x": 42})
15print(child["x"])    # 42 -- from child
16print(child["y"])    # 2  -- from base, falls through
17print(child.parents["x"])  # 1 -- back to parent scope

The difference from dict | dict: | creates a new merged dict. ChainMap keeps the originals separate and reads lazily, so changes to the underlying dicts are visible through the ChainMap. Use | for a snapshot, use ChainMap for a live layered view.