Picking the Right Model — async vs threads vs processes

The decision tree

(1) Is the work I/O-bound (network, files, database)? → asyncio if your stack supports it, threading if it doesn't. (2) Is the work CPU-bound? → multiprocessing. (3) Both? → asyncio for the I/O parts, offload CPU work to a process pool from inside the async function (asyncio.to_thread, loop.run_in_executor).

asyncio.to_thread — bridging sync to async

3.9+ added await asyncio.to_thread(sync_fn, *args). Runs sync_fn in a thread, awaits its completion, returns the result. Perfect for calling a blocking library from async code without blocking the event loop. cwkPippa uses this for the few sync-only operations it has.

The cwkPippa example — what async-all-the-way looks like

cwkPippa is asyncio top to bottom. The web layer (FastAPI) is async. Database access (aiosqlite) is async. The Claude/Codex/Gemini SDKs all expose async streaming. JSONL writes use async file I/O. Every chat request runs as a single async coroutine through the entire stack — no thread pool, no process pool, just the event loop dispatching between thousands of waiting I/Os.

The free-threaded future (3.13+, experimental)

Python 3.13 added an experimental free-threaded build (no GIL). When stable and widely supported, this changes the "CPU-bound use multiprocessing" calculus — threads will actually parallelize. Today (early 2026), this is opt-in and not yet the default. For 99% of code today, the GIL still applies and the multiprocessing-for-CPU answer is correct.

War Story: Pippa's Claude-Pippa adapter once tried asyncio.wait_for around __anext__ for a per-chunk timeout. When the timeout fired during a tool call, the cancellation finalized the async generator and subsequent next-calls raised StopIteration. The fix was timeout-around-the-whole-stream, not per-chunk. Async cancellation is subtle — treat it like load-bearing structural code.

Code

Decision: I/O work — asyncio is the answer·python

import asyncio
import aiohttp                         # async HTTP client

async def fetch(session, url):
    async with session.get(url) as resp:
        return await resp.text()

async def fetch_all(urls):
    async with aiohttp.ClientSession() as session:
        return await asyncio.gather(*(fetch(session, u) for u in urls))

# (Demo only — would run with asyncio.run(fetch_all(urls)))
# Hundreds of concurrent fetches, single thread, one event loop

Decision: CPU work — multiprocessing is the answer·python

from concurrent.futures import ProcessPoolExecutor

def heavy_compute(n):
    total = 0
    for i in range(n):
        total += i * i
    return total

if __name__ == '__main__':
    with ProcessPoolExecutor() as pool:
        results = list(pool.map(heavy_compute, [10**7] * 4))
        print(results)
# 4 cores actually used in parallel, no GIL constraint

Mixed — async with offloaded CPU work·python

import asyncio

def heavy_compute(n):                  # sync, CPU-bound
    total = 0
    for i in range(n):
        total += i * i
    return total

async def main():
    # Offload to a thread (or process) so the event loop isn't blocked
    result = await asyncio.to_thread(heavy_compute, 10_000_000)
    print(result)

asyncio.run(main())

# For process-level: loop.run_in_executor(ProcessPoolExecutor(), fn, *args)

When threading IS the right call·python

# A specific case where threading wins:
# - You have to use a sync library (no async version)
# - You're not refactoring it to async right now
# - The work is I/O-bound
#
# Example: legacy database driver that's sync only.
# Wrap it in asyncio.to_thread or use ThreadPoolExecutor.
#
# Don't blindly default to threading — it's the right answer
# in fewer cases than people assume. asyncio supports nearly
# every modern I/O library; threading is for filling gaps.

Exercise

Decide which model fits each scenario and write a one-sentence justification: (a) Downloading 100 web pages in parallel. (b) Resizing 1000 images on a multi-core machine. (c) A web server handling thousands of websocket connections. (d) Calling a sync database library from inside an async FastAPI endpoint. (e) Computing the Fibonacci sequence up to N=200 (the recursive way, no cache).

Picking the Right Model — async vs threads vs processes

The decision tree

asyncio.to_thread — bridging sync to async

The cwkPippa example — what async-all-the-way looks like

The free-threaded future (3.13+, experimental)

Code

External links

Exercise

Progress

Comments 0