Generator Expressions vs List Comprehensions — When to Choose
~15 min · generator-expression, comprehension, lazy, memory
Level 0Curious
0 XP0/93 lessons0/23 achievements
0/100 XP to next level100 XP to go0% complete
Same syntax, opposite semantics
The only visual difference is brackets. [x*x for x in nums] is a list comprehension — it builds and returns a list. (x*x for x in nums) is a generator expression — it returns a lazy iterator. The performance and memory characteristics are completely different.
When generator expressions win
Three cases. (1) When you only need to consume the values once and feed them into another consumer (sum, any, all, max, min, a for loop). (2) When the source is huge or infinite. (3) When you want to short-circuit — stop processing as soon as some condition is met. In all three, building a list first is wasted work.
When list comprehensions win
When you need to iterate the result more than once, or use it where len(), indexing, or slicing matters, or pass it to something that requires a list (some library APIs). Generators don't support any of those. The moment you find yourself wrapping a generator in list(...) just to enable basic operations, you should have written a list comprehension to begin with.
The dropped-parentheses idiom
When a generator expression is the only argument to a function call, the outer parentheses can be dropped. sum((x*x for x in nums)) is the same as sum(x*x for x in nums). This is the most common form you'll write — most generator expressions live inside another function's parens.
Pythonic Way: The default for "process every element of..." pipelines should be the generator expression. Switch to a list comprehension when you specifically need a list. Most beginners default to list comprehensions and learn the cost the day they iterate a 10GB log file.
Code
Memory difference — measurable·python
import sys
lc = [x*x for x in range(1_000_000)] # builds a million-element list
gen = (x*x for x in range(1_000_000)) # builds a generator object
print(sys.getsizeof(lc)) # ~8MB
print(sys.getsizeof(gen)) # ~200 bytes — generator object is tiny
# Same answers when consumed
print(sum(lc) == sum((x*x for x in range(1_000_000)))) # True
def expensive(x):
print("computing", x)
return x * x
# List comprehension — computes ALL elements first
result_list = any(v > 10 for v in [expensive(x) for x in [1, 2, 3, 4, 5]])
# Prints all 5 'computing' lines BEFORE checking
print("---")
# Generator expression — short-circuits when it finds a match
result_gen = any(expensive(x) > 10 for x in [1, 2, 3, 4, 5])
# Prints just enough — stops at the first hit
When to switch to a list — multiple consumption·python
# This DOESN'T work — generator is exhausted after first use
gen = (x*x for x in range(5))
print(list(gen)) # [0, 1, 4, 9, 16]
print(list(gen)) # [] <- exhausted
print(sum(gen)) # 0 <- still exhausted
# If you need to use it multiple times, use a list comprehension
lst = [x*x for x in range(5)]
print(list(lst)) # [0, 1, 4, 9, 16]
print(list(lst)) # [0, 1, 4, 9, 16]
print(sum(lst)) # 30
Dropped-parentheses idiom·python
nums = [1, 2, 3, 4, 5]
# All three work; the third is the most common form
print(sum((x*x for x in nums))) # 55
print(sum( (x*x for x in nums) )) # 55
print(sum(x*x for x in nums)) # 55 <- standard idiom
# Same with any/all/max/min
print(any(x > 3 for x in nums)) # True
print(max(x*2 for x in nums)) # 10
Read a file (or simulate with a multiline string) of 100 lines. Write a pipeline of generator expressions that: (a) strips whitespace from each line, (b) filters out empty lines, (c) extracts the first word of each non-empty line, (d) yields only words longer than 4 characters. Pass the final pipeline to set() to get unique long-first-words. Verify you never built a full list of all 100 stripped lines anywhere in the pipeline.
Progress
Progress is local-only — sign in to sync across devices.