Picking the Right Model — async vs threads vs processes
~15 min · picking, io-bound, cpu-bound, patterns
Level 0Curious
0 XP0/93 lessons0/23 achievements
0/100 XP to next level100 XP to go0% complete
The decision tree
(1) Is the work I/O-bound (network, files, database)? → asyncio if your stack supports it, threading if it doesn't. (2) Is the work CPU-bound? → multiprocessing. (3) Both? → use asyncio for I/O, asyncio.to_thread for short blocking I/O, and loop.run_in_executor with a process pool for CPU work.
asyncio.to_thread — bridging sync to async
3.9+ added await asyncio.to_thread(sync_fn, *args). Runs sync_fn in a thread, awaits its completion, returns the result. Perfect for calling a blocking library from async code without blocking the event loop. cwkPippa uses this for the few sync-only operations it has.
The cwkPippa example — async-centered, with explicit sync boundaries
cwkPippa's chat routes and brain streams are centered on asyncio, but not every internal operation is asynchronous. JSONL append is one synchronous file boundary. The useful discipline is to identify blocking work and move long operations to a thread or process deliberately, not to repeat “async all the way.”
The free-threaded future (3.13+, experimental)
Python 3.13 added an experimental free-threaded build (no GIL). When stable and widely supported, this changes the "CPU-bound use multiprocessing" calculus — threads will actually parallelize. Today (early 2026), this is opt-in and not yet the default. For 99% of code today, the GIL still applies and the multiprocessing-for-CPU answer is correct.
War Story: Pippa's Claude-Pippa adapter once tried asyncio.wait_for around __anext__ for a per-chunk timeout. When the timeout fired during a tool call, the cancellation finalized the async generator and subsequent next-calls raised StopIteration. The fix was timeout-around-the-whole-stream, not per-chunk. Async cancellation is subtle — treat it like load-bearing structural code.
Code
Decision: I/O work — asyncio is the answer·python
import asyncio
import aiohttp # async HTTP client
async def fetch(session, url):
async with session.get(url) as resp:
return await resp.text()
async def fetch_all(urls):
async with aiohttp.ClientSession() as session:
return await asyncio.gather(*(fetch(session, u) for u in urls))
# (Demo only — would run with asyncio.run(fetch_all(urls)))
# Hundreds of concurrent fetches, single thread, one event loop
Decision: CPU work — multiprocessing is the answer·python
from concurrent.futures import ProcessPoolExecutor
def heavy_compute(n):
total = 0
for i in range(n):
total += i * i
return total
if __name__ == '__main__':
with ProcessPoolExecutor() as pool:
results = list(pool.map(heavy_compute, [10**7] * 4))
print(results)
# 4 cores actually used in parallel, no GIL constraint
Mixed — async with offloaded CPU work·python
import asyncio
def heavy_compute(n): # sync, CPU-bound
total = 0
for i in range(n):
total += i * i
return total
async def main():
# Offload to a thread (or process) so the event loop isn't blocked
result = await asyncio.to_thread(heavy_compute, 10_000_000)
print(result)
asyncio.run(main())
# For process-level: loop.run_in_executor(ProcessPoolExecutor(), fn, *args)
When threading IS the right call·python
# A specific case where threading wins:
# - You have to use a sync library (no async version)
# - You're not refactoring it to async right now
# - The work is I/O-bound
#
# Example: legacy database driver that's sync only.
# Wrap it in asyncio.to_thread or use ThreadPoolExecutor.
#
# Don't blindly default to threading — it's the right answer
# in fewer cases than people assume. asyncio supports nearly
# every modern I/O library; threading is for filling gaps.
Decide which model fits each scenario and write a one-sentence justification: (a) Downloading 100 web pages in parallel. (b) Resizing 1000 images on a multi-core machine. (c) A web server handling thousands of websocket connections. (d) Calling a sync database library from inside an async FastAPI endpoint. (e) Computing the Fibonacci sequence up to N=200 (the recursive way, no cache).
Progress
Progress is local-only — sign in to sync across devices.