The previous lesson ended with an awkward question: threads overlap waits, but each thread costs memory and opens the door to race conditions. What if you had to manage 500 queries? asyncio proposes another model: a single thread that never waits idle. Think of the best waiter in the café next to Papyrus: he takes an order at table 1, and while the kitchen prepares it he doesn't stand staring at the kitchen door — he goes to table 2, serves table 3, takes payment at table 4, and returns to table 1 just as its dish comes out. One waiter, many tables, zero dead time. That's asynchronous programming: tasks declare where they're going to wait (await) and a coordinator (the event loop) uses each wait to advance another task.
Contents
- The idea: a single thread that never waits idle
async defandawait: coroutines- The foundational mistake: calling a coroutine doesn't run it
asyncio.run(): starting up the async worldasyncio.sleepvstime.sleep: the cardinal sinasyncio.gather: N queries at once, measured- Asyncio vs threads: the table
async withandasync for, briefly- Honesty: when it pays off and when it doesn't
- Wrapping up module 8
async def and await: coroutines
A function declared with async def is a coroutine: a function that can pause at its waits and yield its turn. If in 08-03 you saw that a generator pauses at each yield, you already have the exact intuition — coroutines were historically built on that same pause-and-resume machinery:
import asyncio
async def fetch_price(distributor: str, title: str) -> tuple[str, float]:
print(f"→ querying {title} at {distributor}...")
await asyncio.sleep(2) # "I'm going to wait 2 s here: use the turn"
prices = {"Distributor A": 19.90, "Distributor B": 21.50,
"Distributor C": 20.10}
print(f"← {distributor} responds")
return distributor, prices[distributor]await marks the waiting points: "I stop here; event loop, serve someone else and wake me when this is ready". As in 08-05, asyncio.sleep(2) simulates the network wait of a real query; it's still the standard way to learn this without servers.
The foundational mistake: calling it doesn't run it
Just as read_sales(path) in 08-03 read nothing (it returned a paused generator), calling a coroutine does not run its body:
result = fetch_price("Distributor A", "Faust")
print(result)
# <coroutine object fetch_price at 0x...> ← nothing has been queried!
# RuntimeWarning: coroutine 'fetch_price' was never awaitedThis is the classic asyncio mistake: forgetting the await. The call creates the coroutine object; running it requires await (from another coroutine) or handing it to the event loop. If you see RuntimeWarning: ... was never awaited in your terminal, look for the call without await — there always is one.
asyncio.run(): starting up the async world
await can only be written inside an async def... so who runs the first coroutine? asyncio.run(): it creates the event loop (the waiter), hands it the main coroutine and doesn't return until everything finishes:
async def main() -> None:
dist, price = await fetch_price("Distributor A", "Faust")
print(f"{dist}: {price} EUR")
asyncio.run(main()) # the ONLY entry point; once per programThe standard structure of an async script: coroutines at the top, an asynchronous main() that orchestrates them, and a single asyncio.run(main()) at the end (under if __name__ == "__main__": if it's a module, as always since M3).
asyncio.sleep vs time.sleep: the cardinal sin
Here is the rule that separates asyncio that works from asyncio that doesn't. The event loop is a single thread: if a coroutine runs something blocking — time.sleep(2), a giant open().read(), a synchronous HTTP request — the waiter stands frozen at that table and all the other tasks stop with him.
async def query_BAD(distributor: str) -> None:
time.sleep(2) # ☠ blocks the ENTIRE event loop: nobody else advances
async def query_GOOD(distributor: str) -> None:
await asyncio.sleep(2) # ✔ yields the turn: the others advance while I waittime.sleep(2) |
await asyncio.sleep(2) |
|
|---|---|---|
| Who waits? | The whole thread (the waiter, standing still) | Only that coroutine (that table) |
| The other tasks | Frozen | Advance |
| 3 "concurrent" queries | 6 s (sequential in disguise!) | 2 s |
An important generalization: everything you use inside asyncio must be async. That's why the ecosystem has parallel libraries: for real HTTP the classic requests won't do (synchronous, blocks the loop) — you use aiohttp, which we only mention here; real requests arrive in module 10.
asyncio.gather: N queries at once
await-ing one at a time is still sequential (wait for A, then wait for B...). To launch several coroutines at once and collect all the results, asyncio.gather:
import time
async def main() -> None:
distributors = ["Distributor A", "Distributor B", "Distributor C"]
# Sequential version: each await waits for the previous one to finish
start = time.perf_counter()
for d in distributors:
await fetch_price(d, "Faust")
print(f"sequential: {time.perf_counter() - start:.1f} s") # ~6.0 s
# gather version: the three waits overlap on the same thread
start = time.perf_counter()
results = await asyncio.gather(
*(fetch_price(d, "Faust") for d in distributors) # genexpr from 08-03
)
print(f"gather: {time.perf_counter() - start:.1f} s") # ~2.0 s
dist, price = min(results, key=lambda r: r[1])
print(f"cheapest: {dist} at {price} EUR")
asyncio.run(main())gather takes coroutines (here unpacked with M3's * from a generator expression), runs them concurrently and returns the list of results in input order — no threads, no Lock, no race conditions: between one await and the next nobody interrupts you, so the sales counter from 08-05 wouldn't get corrupted here. If a coroutine raises, gather propagates the exception (and with return_exceptions=True it hands them to you as values so you can handle them one by one, in the style of the futures from 08-05).
Asyncio vs threads
Same time (~2 s) in the distributors example — so which one should you use?
| Criterion | Threads (ThreadPoolExecutor) |
asyncio |
|---|---|---|
| Model | Several threads; the OS decides the turns | One thread; tasks yield their turn at await |
| Race conditions | Possible on any line → Lock |
Only at the awaits (much more predictable) |
| Reasonable scale | Dozens of tasks | Thousands of tasks (one connection = one cheap coroutine) |
| Works with normal (synchronous) libraries? | Yes — its great advantage | No: it demands async libraries (aiohttp, not requests) |
| Speeds up pure CPU? | No (GIL) | No (a single thread!) — for CPU, processes (08-05) |
| Code style | Normal functions | async/await everywhere (it's contagious) |
Practical rule: a few I/O tasks with classic libraries → threads, which fit in without rewriting anything. A huge number of simultaneous connections, or an already-async ecosystem → asyncio. CPU → processes, always.
async with and async for
There are asynchronous versions of two old friends: async with for context managers whose entry/exit waits on I/O (a network connection that's slow to open — the protocol is __aenter__/__aexit__, a mirror of 08-04), and async for for iterating over sources that produce values with a wait between them. It's enough to recognize them when you see them in async library documentation; you won't need them until you work with those libraries.
Honesty: when it pays off
Let's be clear, because async comes with a lot of marketing: for Papyrus — a script that queries 3 distributors twice a day — asyncio does not pay off. The ThreadPoolExecutor of 08-05 solves the same thing, with fewer new concepts and without demanding special libraries. Even the 6-second sequential version is defensible: it's the simplest, and 6 seconds twice a day doesn't hurt (the last row of the 08-05 decision table still applies).
Where asyncio truly shines is where threads can't reach: servers with thousands of simultaneous connections — a thousand clients waiting for a response are a thousand cheap coroutines sleeping at their awaits, not a thousand threads devouring memory. That's why Python's modern web frameworks are asynchronous inside. When we build web services for Papyrus in module 10, you'll know what engine roars underneath.
Common Mistakes and Tips
- Forgetting the
await: you get a coroutine object and aRuntimeWarning: coroutine ... was never awaited. The function "does nothing". It's mistake #1; now you know how to read its symptom. time.sleep(or any blocking call) inside a coroutine: it freezes the whole event loop and your "concurrency" becomes sequential in disguise. Inside anasync def, every wait is done withawait.awaitin series believing it's concurrent:await a(); await b()runs a and then b. The concurrency is created bygather(orasyncio.TaskGroup, its modern alternative), not by the wordasync.- Calling
asyncio.run()inside a coroutine (or nesting it twice):RuntimeError. A singlerun(), at the program's boundary — like thebasicConfigof 07-05, which is also configured just once at the entry point. - Mixing
requestsor heavyopen()calls into coroutines: they work, but they block the loop. Async ecosystem or threads; not half and half. - Tip: when in doubt between threads and asyncio, start with threads. Migrating to asyncio later is manageable; forcing asyncio where it doesn't belong means rewriting everything.
Exercises
-
Write
async def download_cover(title: str) -> strthat simulates the download withawait asyncio.sleep(1)and returnsf"{title}.webp". Download the covers of the four books in the catalog (The Odyssey, Hamlet, Don Quixote, Faust) withgatherand check by measuring withtime.perf_counter()that it takes ~1 s and not ~4 s. -
This code claims to take 2 s and takes 6. Explain why, line by line, and fix it (two changes).
async def query(d): time.sleep(2) return d async def main(): results = [await query(d) for d in ("A", "B", "C")] asyncio.run(main()) -
Without writing code: for each Papyrus scenario, choose sequential, threads, processes or asyncio, and justify it in one sentence. (a) Generating the annual sales statistics for 10 years, computation-heavy. (b) The daily till close, which takes 0.04 s. (c) A future web service serving 2,000 neighborhood bookshops checking stock at the same time.
Solutions
-
import asyncio, time async def download_cover(title: str) -> str: await asyncio.sleep(1) # simulated download return f"{title}.webp" async def main() -> None: titles = ["The Odyssey", "Hamlet", "Don Quixote", "Faust"] start = time.perf_counter() covers = await asyncio.gather(*(download_cover(t) for t in titles)) print(covers, f"{time.perf_counter() - start:.1f} s") # [...] 1.0 s asyncio.run(main())Four 1 s waits overlapped on a single thread: total ~1 s.
-
Two flaws:
time.sleep(2)blocks the event loop (nobody advances during those 2 s), and the comprehension withawait query(d)waits for each query before launching the next — pure sequential. Fix:async def query(d): await asyncio.sleep(2) # change 1: async wait that yields the turn return d async def main(): results = await asyncio.gather(*(query(d) for d in ("A", "B", "C"))) # change 2: gather launches all three at once → ~2 s -
(a) Processes (
ProcessPoolExecutor): it's pure CPU and neither threads nor asyncio speed it up — only the real parallelism of several GILs. (b) Sequential: 0.04 s isn't a wait that hurts; adding concurrency would be paying complexity for nothing. (c) Asyncio: thousands of simultaneous I/O connections is exactly the scenario where cheap coroutines beat threads — and the home turf of the module 10 frameworks.
Conclusion
asyncio completes the concurrency map: a single thread, coroutines that pause at await the way generators did at yield, asyncio.run() as the one entry door, the golden rule of never blocking the event loop (asyncio.sleep, never time.sleep), and gather turning 6 seconds into 2 without a single Lock. And the mature lesson: for Papyrus's script, threads — or nothing — are enough; async shines in servers with thousands of connections, an appointment we'll keep in module 10.
Module 8 delivered, one by one, on the promises made at the close of module 7. The contracts that module defended by hand are now written into the signatures — find_book(catalog: dict[str, Book], title: str) -> Book | None tells the whole truth, and mypy verifies it without running anything (08-01). The repeated plumbing moved out of the functions and became wrappers: @timed measures, @log_call audits in papyrus.log and @retry(times=3) perseveres, all without touching a line of sell() (08-02). The till close stopped loading the whole of sales.csv: a generator pipeline — read → parse → filter → sum — processes a year of sales with the memory of a single row (08-03). The oldest promise, the with we'd been dragging along since module 6, was settled: __enter__ prepares, __exit__ always cleans up, and CatalogTransaction guarantees that a half-finished sale never leaves the catalog corrupt (08-04). And when the waits started to hurt, you learned to overlap them with threads, to parallelize computation with processes and to orchestrate thousands of waits with a single waiter — knowing, above all, when none of the three is needed (08-05, 08-06). Papyrus is now expressive, efficient and ready to grow. But one question remains that no type hint, decorator or context manager answers: when you change a line tomorrow — a new discount, one more field in the CSV — how do you guarantee that all of this STILL works? Checking by hand every time doesn't scale. You write code that checks the code: tests. That's module 9.
Python Programming Course
Module 1: Introduction to Python
- Introduction to Python
- Setting Up the Development Environment
- Python Syntax and Basic Data Types
- Variables and Constants
- Basic Input and Output
- Virtual Environments and Package Management
Module 2: Control Structures
Module 3: Functions and Modules
- Defining Functions
- Function Arguments
- Lambda Functions
- Modules and Packages
- Standard Library Overview
Module 4: Data Structures
Module 5: Object-Oriented Programming
Module 6: File Handling
Module 7: Error and Exception Handling
- Introduction to Exceptions
- Handling Exceptions
- Raising Exceptions
- Custom Exceptions
- Best Practices and Error Logging
Module 8: Advanced Topics
- Type Hints
- Decorators
- Generators
- Context Managers
- Concurrency: Threads and Processes
- Asyncio for Asynchronous Programming
Module 9: Testing and Debugging
- Introduction to Testing
- Unit Testing with unittest
- Testing with pytest
- Test-Driven Development
- Debugging Techniques
- Using pdb for Debugging
Module 10: Web Development with Python
- Introduction to Web Development
- Flask Framework Fundamentals
- Building REST APIs with Flask
- Introduction to Django
- Building Web Applications with Django
Module 11: Data Science with Python
- Introduction to Data Science
- NumPy for Numerical Computing
- Pandas for Data Manipulation
- Matplotlib for Data Visualization
- Introduction to Machine Learning with scikit-learn
