This is the lesson where we settle the oldest debt in the course. Since module 6 you've been writing with path.open() as f: on the strength of two promises: "the file closes itself, no matter what" and "one day we'll see how it works inside". Module 7 added the key clue — try/finally guarantees cleanup even with exceptions — and the previous lesson showed you a with inside a generator. Today the circle closes: the __enter__/__exit__ protocol behind every with, and with it we'll build Papyrus's own context managers, including a CatalogTransaction that leaves the catalog safe if a sale breaks down halfway through.
Contents
- What
withreally does - The protocol:
__enter__and__exit__ - Reimplementing (conceptually) the file
with - Our own manager as a class:
CatalogTransaction __exit__and exceptions: propagate or suppresscontextlib.@contextmanager: the generator version- Other uses: timing a block, changing directory
- Class vs
@contextmanager: decision table
What with really does
Let's start with what with is not: it's not a magic "for files" block. It's syntactic sugar over a try/finally pattern you could already write by hand with what you learned in 07-02:
# What you write:
with (BASE / "data" / "catalog.json").open(encoding="utf-8") as f:
data = f.read()
# What Python executes (essential equivalent):
f = (BASE / "data" / "catalog.json").open(encoding="utf-8")
f_ctx = f.__enter__() # prepare the context; its return value goes to the "as"
try:
data = f_ctx.read()
finally:
f.__exit__(None, None, None) # ALWAYS clean up: on success or on exceptionAny object that implements those two dunder methods — and you know about dunders from module 5 — can be used with with. It's called a context manager: it manages entering and leaving a context, guaranteeing the exit.
The protocol: __enter__ and __exit__
| Method | When it's called | What it does | What it returns |
|---|---|---|---|
__enter__(self) |
On entering the with |
Acquire/prepare the resource | Whatever you want bound to the as (often self) |
__exit__(self, exc_type, exc_val, exc_tb) |
On leaving, always | Release/clean up | True → suppresses the exception; False/None → propagates it |
The three arguments of __exit__ describe what happened inside the block:
- If the block finished cleanly: all three are
None. - If the block raised:
exc_typeis the exception class (e.g.InsufficientStockError),exc_valthe instance (with itstitle,requested,availableattributes from 07-04) andexc_tbthe traceback.
In other words: __exit__ is a finally with superpowers — not only does it always run, it also knows whether there was an exception and which one.
Reimplementing the file with
To pin the concept down, let's write our own conceptual "open" (the real one is in C, but the protocol is identical):
class File:
"""Didactic reimplementation of the open() context manager."""
def __init__(self, path, mode="r", encoding="utf-8"):
self.path = path
self.mode = mode
self.encoding = encoding
self.f = None
def __enter__(self):
print(f"[enter] opening {self.path}")
self.f = open(self.path, self.mode, encoding=self.encoding)
return self.f # this is what "as f" receives
def __exit__(self, exc_type, exc_val, exc_tb):
print(f"[exit] closing {self.path} (exception: {exc_type})")
self.f.close() # no matter what
return False # we suppress nothing: let it propagate
with File(BASE / "data" / "additions_log.txt") as f:
print(f.readline())If something raises inside the block, you'll see [exit] closing ... (exception: <class '...'>) before the traceback reaches your screen: the cleanup happened, and the exception continued on its way up the rings of 07-04. Exactly the two promises of with, now without mystery.
CatalogTransaction: the real Papyrus case
Now a manager that solves a genuine problem. sell() deducts stock, then the sale is recorded and the catalog saved. What happens if Ana sells Marta a batch of three titles and the second one raises InsufficientStockError? The first was already deducted: the catalog is left halfway, in a state that matches no real sale. We want transaction semantics: all or nothing.
import copy
import logging
logger = logging.getLogger(__name__)
class CatalogTransaction:
"""If the block fails, restores the catalog to its initial state and re-raises."""
def __init__(self, catalog: dict):
self.catalog = catalog
self.snapshot = None
def __enter__(self):
self.snapshot = copy.deepcopy(self.catalog) # photo of the initial state
return self.catalog # you work on the original
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is not None: # something broke inside
self.catalog.clear()
self.catalog.update(self.snapshot) # restore the photo
logger.warning("transaction rolled back by %s: %s",
exc_type.__name__, exc_val)
return False # False → the exception propagatesdeepcopy (the deep cousin of the copy we brushed against in M4) photographs the catalog and the books; restoring by mutating with clear()/update() — instead of reassigning — keeps every other reference to the same dictionary valid. Its use at the counter:
from warehouse import sell
from errors import PapyrusError
batch = [("Hamlet", 2), ("The Odyssey", 9), ("Faust", 1)] # The Odyssey: only 4 left
try:
with CatalogTransaction(catalog) as cat:
for title, units in batch:
sell(cat, title, units)
except PapyrusError as e:
print(f"Sale cancelled, the catalog is intact: {e}")Sequence: 2 copies of Hamlet are sold (stock 6 → 4)... and The Odyssey, with 9 units requested and 4 available, raises InsufficientStockError. __exit__ sees it coming, restores the photo — Hamlet goes back to 6 — leaves a WARNING in papyrus.log, and returns False: the exception keeps climbing until it reaches the counter's except PapyrusError, which reports it with the calm of 07-04. Nobody lost data; nobody silenced anything.
Propagate or suppress: the return value of __exit__
The value returned by __exit__ is a delicate switch:
False(orNone, which is what a method without areturngives back): the exception propagates. That's the right choice in 95% of cases — the manager cleans up, it doesn't decide.True: the exception is suppressed, as if the block had gone well. It only makes sense when suppressing is the service you offer (e.g.contextlib.suppress(FileNotFoundError), which the standard library already ships ready-made).
A careless return True is the except: pass of 07-04 dressed up as elegance. When in doubt, False.
contextlib.@contextmanager: the generator version
Writing a whole class just for "do something before, do something after" can be a lot of ceremony. contextlib offers a shortcut that brings the two previous lessons together — a decorator applied to a generator:
from contextlib import contextmanager
@contextmanager
def catalog_transaction(catalog: dict):
snapshot = copy.deepcopy(catalog) # ── this plays the role of __enter__
try:
yield catalog # ── the with block runs here
except Exception:
catalog.clear()
catalog.update(snapshot) # ── this plays __exit__ with an exception
raise # re-raising = the class's "return False"The exact correspondence:
- Everything before the
yieldis the__enter__. - The
yield's value is what theasreceives. - The generator stays paused at the
yield(just as you saw in 08-03) while the body of thewithruns. - If the block raises, the exception surfaces at the
yieldpoint — that's why it's wrapped intry/except(ortry/finallyif there's only cleanup to do). - Resuming and finishing quietly amounts to propagating normality;
raisepropagates the exception; catching it without re-raising amounts toreturn True.
The same behavior as the class, in a third of the lines.
Other uses: not just resources
Any "set up / undo" pattern fits. Two useful examples in Papyrus:
import time
from contextlib import contextmanager
@contextmanager
def stopwatch(label: str):
"""The block-level cousin of @timed from 08-02: it measures a CHUNK of code."""
start = time.perf_counter()
try:
yield
finally:
logger.info("%s: %.3f s", label, time.perf_counter() - start)
with stopwatch("close + report"):
total = close_till(BASE / "data" / "sales.csv")
generate_report(total)import os
from pathlib import Path
@contextmanager
def in_directory(path: Path):
"""Changes the cwd temporarily and ALWAYS comes back (handy in reports/)."""
previous = Path.cwd()
os.chdir(path)
try:
yield path
finally:
os.chdir(previous) # even if generating the report blows upNotice that stopwatch does with a block what @timed did with a function: decorators and context managers are the two faces of "wrapping code".
Class vs @contextmanager
| Criterion | Class (__enter__/__exit__) |
@contextmanager |
|---|---|---|
| Lines of code | More (class ceremony) | Fewer (one generator) |
| Complex state or extra methods | Natural (attributes, methods) | Awkward |
| Reusable/configurable via inheritance | Yes | Not applicable |
| Distinguishing exception types on exit | Explicit exc_type |
except SomeType: inside the generator |
| Readability for simple cases | Fine | Excellent |
| Ideal example | CatalogTransaction with options |
stopwatch, in_directory |
Practical rule: start with @contextmanager; move to the class when you need state, configuration or a hierarchy.
Common Mistakes and Tips
- Forgetting the
tryaround theyieldin a@contextmanager: if the block raises, your cleanup code doesn't run — the generator dies at theyield. Thetry/finally(ortry/except+raise) isn't optional: it's the equivalent of__exit__. return True(or catching without re-raising) by accident: it suppresses exceptions the counter was expecting. Typical symptom: "the sale failed but no error ever appeared". Check__exit__'s return value before anything else.- Doing heavy work in
__init__instead of__enter__:CatalogTransaction(catalog)without awithshouldn't take any snapshot yet.__init__stores parameters;__enter__acquires. - Shallow-copying when there are nested objects:
dict(catalog)copies the dictionary but not theBookobjects; the rollback would leave corrupt stocks. For transactions,copy.deepcopy. - Tip: if you write
try/finallyaround a resource more than once, that's the sign that a context manager with a name of its own lives there. - Tip: they can be chained in a single
with:with catalog_transaction(cat) as c, stopwatch("sale"):— they exit in the reverse order they were entered.
Exercises
- Write as a class the manager
BlockLog(name)that on entry logsSTART <name>and on exit logsEND <name> (ok)orEND <name> (error: <ExcType>)depending on whether there was an exception, always propagating it. Test it by wrapping a sale that raisesInsufficientStockError. - Rewrite
BlockLogwith@contextmanager(call itblock_log). Which construct replaces theif exc_type is not Nonecheck? - Write
make_backup(path)with@contextmanager: before the block, it copies the file tobackups/<name>.bak(useshutil.copy2, from M6); if the block fails, it restores the.bakover the original and re-raises; if all goes well, it does nothing more. Use it aroundsave_catalog.
Solutions
-
class BlockLog: def __init__(self, name: str): self.name = name def __enter__(self): logger.info("START %s", self.name) return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is None: logger.info("END %s (ok)", self.name) else: logger.error("END %s (error: %s)", self.name, exc_type.__name__) return False # always propagate with BlockLog("Marta batch sale"): sell(catalog, "The Odyssey", 9) # raises InsufficientStockErrorpapyrus.logends up withSTART+END ... (error: InsufficientStockError), and the exception continues on to the counter. -
@contextmanager def block_log(name: str): logger.info("START %s", name) try: yield logger.info("END %s (ok)", name) except Exception as e: logger.error("END %s (error: %s)", name, type(e).__name__) raiseThe
if exc_type is not Nonecheck turns into thetry/exceptstructure: the exception-free path continues after theyield; the exception path enters through theexcept— and the finalraisepreserves propagation. -
import shutil from contextlib import contextmanager @contextmanager def make_backup(path: Path): backup = BASE / "backups" / (path.name + ".bak") shutil.copy2(path, backup) try: yield path except Exception: shutil.copy2(backup, path) # restore the original raise with make_backup(BASE / "data" / "catalog.json") as path: save_catalog(catalog, path)It's
CatalogTransactiontaken to disk: the same idea (photo → try → restore on failure → re-raise) applied to a file instead of a dictionary.
Conclusion
Promise settled: with was a two-dunder protocol — __enter__ prepares, __exit__ always cleans up and decides with its return value whether the exception propagates (almost always False) — built on the same try/finally from 07-02. Papyrus gained CatalogTransaction, which guarantees all-or-nothing sales, and the @contextmanager versions proved that decorators (08-02) and generators (08-03) weren't loose topics: here they work together. With this, Papyrus's code is expressive and efficient... but it still does one thing at a time: if asking one distributor for a book's price takes two seconds, asking three takes six, with the program standing idle waiting. The module's last two lessons attack that waiting: first with threads and processes — and an honest conversation about the famous GIL — and then with asyncio.
Python Programming Course
Module 1: Introduction to Python
- Introduction to Python
- Setting Up the Development Environment
- Python Syntax and Basic Data Types
- Variables and Constants
- Basic Input and Output
- Virtual Environments and Package Management
Module 2: Control Structures
Module 3: Functions and Modules
- Defining Functions
- Function Arguments
- Lambda Functions
- Modules and Packages
- Standard Library Overview
Module 4: Data Structures
Module 5: Object-Oriented Programming
Module 6: File Handling
Module 7: Error and Exception Handling
- Introduction to Exceptions
- Handling Exceptions
- Raising Exceptions
- Custom Exceptions
- Best Practices and Error Logging
Module 8: Advanced Topics
- Type Hints
- Decorators
- Generators
- Context Managers
- Concurrency: Threads and Processes
- Asyncio for Asynchronous Programming
Module 9: Testing and Debugging
- Introduction to Testing
- Unit Testing with unittest
- Testing with pytest
- Test-Driven Development
- Debugging Techniques
- Using pdb for Debugging
Module 10: Web Development with Python
- Introduction to Web Development
- Flask Framework Fundamentals
- Building REST APIs with Flask
- Introduction to Django
- Building Web Applications with Django
Module 11: Data Science with Python
- Introduction to Data Science
- NumPy for Numerical Computing
- Pandas for Data Manipulation
- Matplotlib for Data Visualization
- Introduction to Machine Learning with scikit-learn
