A generator function looks like a regular function but contains yield. Calling it doesn't run the body — it returns a generator object (which is an iterator). Each call to next() runs the body until the next yield, returns that value, and suspends right there. The next next() picks up exactly where it left off, with all local state preserved. When the function returns (or hits the end), StopIteration is raised.
Why generators replace __iter__/__next__ so often
The CountDown class from lesson 1 takes 8 lines plus boilerplate. A generator version takes 3. Same behavior, vastly less code. The trade is that you lose the ability to attach extra methods to your iterator — but you almost never need that. For 90% of iterator use cases, a generator function is the right tool.
Lazy evaluation — the killer feature
A generator only computes values when asked. def big_range(): yield from range(10**10) doesn't allocate memory for ten billion numbers. any(x > 100 for x in big_range()) stops at the first match without ever materializing the whole sequence. Lazy iteration is what makes Python practical for streaming, infinite sequences, and pipelines that would otherwise need a database.
yield from — delegating to another iterable
yield from other is shorthand for for x in other: yield x, but it also forwards send, throw, and return values cleanly. The main use: composing generators. If your generator wants to yield from another generator, yield from is the clean way.
send, throw, close — generator as coroutine
Generators can receive values too. g.send(value) resumes the generator with value as the result of the last yield. g.throw(ExcType) raises an exception at the suspension point. g.close() raises GeneratorExit there. This was once Python's only way to write coroutines — before async/await took over. You'll see it less today, but you should recognize it.
Principle: When you have a function that builds a list and then returns it, ask whether the caller really needs the whole list, or whether they'll consume it one element at a time. If the latter, swap append for yield. Less memory, faster start, same caller-side code.
Code
Generator basics — yield, then suspend·python
def countdown(n):
while n > 0:
yield n
n -= 1
for x in countdown(3):
print(x)
# 3
# 2
# 1
# Same behavior as the CountDown class, in 3 lines instead of 8
print(list(countdown(5))) # [5, 4, 3, 2, 1]
print(sum(countdown(10))) # 55
Lazy evaluation — infinite sequences·python
def naturals():
n = 1
while True:
yield n
n += 1
# Don't iterate everything — just take the first 5
import itertools
first_5 = list(itertools.islice(naturals(), 5))
print(first_5) # [1, 2, 3, 4, 5]
# Stop as soon as we find the answer
result = next(x for x in naturals() if x * x > 1000)
print(result) # 32 (32*32 = 1024)
yield from — delegating·python
def first_half(items):
yield from items[:len(items)//2]
def whole(items):
yield from first_half(items)
yield "middle marker"
yield from items[len(items)//2:]
for x in whole([1, 2, 3, 4, 5, 6]):
print(x)
# 1
# 2
# 3
# middle marker
# 4
# 5
# 6
Streaming — process a large file without loading it all·python
def lines_of(path):
with open(path) as f:
for line in f:
yield line.rstrip("\n")
# Memory cost is one line at a time, no matter how big the file
# (Real example would point at an actual file)
# for line in lines_of("huge.log"):
# if "ERROR" in line:
# print(line)
# Filter pipelines compose naturally
def errors_only(lines):
for line in lines:
if "ERROR" in line:
yield line
# errors_only(lines_of("huge.log")) # still streams, no list ever built
send / throw — generator as coroutine (older style)·python
def echo():
while True:
received = yield
print("got:", received)
g = echo()
next(g) # prime — advance to first yield
g.send("hello") # 'got: hello'
g.send("world") # 'got: world'
g.close() # tells the generator to terminate
Write a generator take_until(predicate, iterable) that yields items from iterable until (and not including) the first item for which predicate(item) is True. Then it stops. (a) Demo with take_until(lambda x: x > 5, range(100)) — should yield 0, 1, 2, 3, 4, 5. (b) Show it works with infinite generators by combining it with the naturals() generator above. (c) Use yield from somewhere in a small chain(*generators) function that flattens multiple generators into one.
Progress
Progress is local-only — sign in to sync across devices.