functools — cache, partial, reduce, wraps · Python Quest

The function-shaped utilities

If itertools is for iterables, functools is for functions. Caching results, currying arguments, reducing sequences, preserving metadata across decorator wrappers. The four worth knowing first: cache/lru_cache, partial, reduce, and wraps (covered in decorators track).

cache and lru_cache — memoization for free

@functools.cache (3.9+) caches results based on the arguments — same args, return the cached value. @lru_cache(maxsize=128) is the same with a bounded LRU cache. Pure functions become dramatically faster for repeated calls. The classic demo: recursive Fibonacci goes from exponential to linear.

partial — pre-binding arguments

functools.partial(fn, *args, **kwargs) returns a new callable with some arguments pre-filled. Useful for adapting a function to an interface that expects fewer arguments — passing it as a callback, sorting key, or event handler with the configuration baked in.

reduce — fold left over an iterable

functools.reduce(fn, iterable) applies fn cumulatively from the left, reducing the iterable to a single value. reduce(operator.add, [1, 2, 3]) is ((1+2)+3). Most uses of reduce are now better expressed with sum, min, max, or a comprehension. Reach for it when you need a custom combining function.

War Story: cache and lru_cache work by hashing the arguments. If your function takes unhashable types (lists, dicts), the cache decorator raises. Solution: use a wrapper that converts to a hashable form, or use cachetools which has more options.

A cache trades time for state. If a function's answer depends on external time, settings, or files not represented in the cache key, it will return wrong answers faster. Memoize only when you can name all meaningful inputs and the invalidation rule.

Code

cache and lru_cache — memoization·python

from functools import cache, lru_cache
import time

# Without cache — recursive Fibonacci is exponential
def fib_slow(n):
    if n < 2:
        return n
    return fib_slow(n - 1) + fib_slow(n - 2)

# With cache — linear
@cache
def fib_fast(n):
    if n < 2:
        return n
    return fib_fast(n - 1) + fib_fast(n - 2)

# fib_slow(35) takes ~3 seconds
# fib_fast(35) is instant
print(fib_fast(50))            # 12586269025  — works for huge n

# lru_cache with size limit
@lru_cache(maxsize=128)
def expensive(x):
    print("computing", x)
    return x * x

print(expensive(3))            # computing 3, returns 9
print(expensive(3))            # cached, no print
print(expensive.cache_info())  # CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)

partial — pre-binding arguments·python

from functools import partial

def multiply(x, y):
    return x * y

double = partial(multiply, 2)         # x=2 pre-bound
print(double(5))                      # 10
print(double(10))                     # 20

# Useful for callbacks
def on_event(event_type, payload):
    print(f"got {event_type}: {payload}")

login_handler = partial(on_event, "login")
login_handler({"user": "alice"})      # got login: {'user': 'alice'}

# Pre-binding kwargs
import json
pretty = partial(json.dumps, indent=2, sort_keys=True)
print(pretty({"b": 2, "a": 1}))

reduce — when sum/min/max aren't enough·python

from functools import reduce
import operator

# Most reduces have built-in equivalents
print(sum([1, 2, 3, 4]))                                # 10
print(reduce(operator.add, [1, 2, 3, 4]))              # 10  (same)

# Where reduce earns its keep — custom combiner
products = reduce(operator.mul, [1, 2, 3, 4], 1)       # 24
print(products)

# Custom function — find longest string
words = ["hi", "hello", "hey", "howdy"]
longest = reduce(lambda a, b: a if len(a) >= len(b) else b, words)
print(longest)                                          # 'howdy'

# But max() with key= is usually clearer
print(max(words, key=len))                              # 'howdy'

cache + unhashable arguments — the gotcha·python

from functools import cache

@cache
def sum_list(items):
    return sum(items)

try:
    sum_list([1, 2, 3])      # list is unhashable
except TypeError as e:
    print("caught:", e)

# Workaround — convert to a hashable type at the boundary
@cache
def sum_tuple(items_tuple):
    return sum(items_tuple)

print(sum_tuple((1, 2, 3)))   # works

# Or write a wrapper
def sum_list_v2(items):
    return sum_tuple(tuple(items))

Exercise

(a) Decorate a recursive function that computes binomial coefficients (C(n, k) = C(n-1, k-1) + C(n-1, k)) with @cache and confirm large inputs are fast. Print cache_info() to see hits/misses. (b) Use partial to create a log_warn function that's print pre-bound with [WARN] as the first argument. (c) Use reduce with a custom function to compute the product of a list — then compare to math.prod.

functools — cache, partial, reduce, wraps

The function-shaped utilities

cache and lru_cache — memoization for free

partial — pre-binding arguments

reduce — fold left over an iterable

Code

External links

Exercise

Progress

Comments 0