PyPython· Dojo

Foundations Updated 2026-09 View as Markdown

Files and Context Managers

Reading and writing files, and the `with` statement that guarantees cleanup — the single most idiomatic construct in Python.

python
with open("notes.txt", "r", encoding="utf-8") as f:
    content = f.read()
# the file is closed here — even if an exception was raised

The with statement is the point. It guarantees the file is closed on every path out of the block, including an exception, which is why you should essentially never call open() without it.

Modes#

ModeMeaning
"r"read (default); error if the file is missing
"w"write; truncates an existing file
"a"append
"x"create; error if it already exists
"rb" / "wb"binary — no decoding, gives you bytes
"r+"read and write

"x" is underused: it is the mode that refuses to overwrite, which is what you want for anything you are creating for the first time.

Reading#

python
with open("data.txt", encoding="utf-8") as f:
    content = f.read()          # the whole file as one string

with open("data.txt", encoding="utf-8") as f:
    for line in f:              # one line at a time — lazy, no size limit
        print(line.rstrip("\n"))

Iterate the file object rather than calling .readlines(). Iteration streams; readlines() loads everything into memory, which is fine for a config file and fatal for an 8GB log.

python
with open("big.log", encoding="utf-8") as f:
    errors = sum(1 for line in f if "ERROR" in line)   # constant memory

Writing#

python
with open("out.txt", "w", encoding="utf-8") as f:
    f.write("first line\n")               # write does NOT add a newline
    f.writelines(f"{n}\n" for n in range(3))
    print("via print", file=f)            # print does add one

pathlib#

The modern way to handle paths. Prefer it to os.path string juggling:

python
from pathlib import Path

config = Path("config") / "settings.json"      # / joins, cross-platform
config.exists()
config.suffix                                   # ".json"
config.stem                                     # "settings"
config.parent                                   # Path("config")

text = config.read_text(encoding="utf-8")       # open, read, close
config.write_text(data, encoding="utf-8")
raw = config.read_bytes()

Path("output").mkdir(parents=True, exist_ok=True)

for py in Path("src").rglob("*.py"):            # recursive glob
    print(py)

read_text and write_text handle the open and close for you, which makes with unnecessary for small whole-file operations.

Writing safely#

A crash partway through a "w" write leaves a truncated file. Write to a temporary file and rename:

python
import os, tempfile
from pathlib import Path

def write_atomic(path: Path, data: str) -> None:
    fd, tmp = tempfile.mkstemp(dir=path.parent, suffix=".tmp")
    try:
        with os.fdopen(fd, "w", encoding="utf-8") as f:
            f.write(data)
            f.flush()
            os.fsync(f.fileno())      # actually on disk
        os.replace(tmp, path)         # atomic within a filesystem
    except BaseException:
        os.unlink(tmp)
        raise

os.replace is atomic, so a reader sees either the old file or the new one — never a half-written one.

What with actually does#

Any object implementing __enter__ and __exit__ works with with. It is a general resource-management protocol, not a file feature:

python
with open("a.txt") as f: ...           # closes the file
with lock: ...                          # releases the lock
with conn.transaction(): ...            # commits or rolls back
with tempfile.TemporaryDirectory() as d: ...   # removes the directory

Several at once:

python
with open("in.txt") as src, open("out.txt", "w") as dst:
    dst.write(src.read())

Writing your own#

The decorator form covers almost every case:

python
from contextlib import contextmanager
import time

@contextmanager
def timed(label: str):
    start = time.perf_counter()
    try:
        yield                                   # the body of the `with` runs here
    finally:
        print(f"{label}: {time.perf_counter() - start:.3f}s")

with timed("import"):
    process_everything()

Everything before yield is setup, everything after is cleanup, and finally guarantees the cleanup runs even when the body raises. That try/finally is the part people forget, and without it the context manager silently stops cleaning up on the exact path where cleanup matters most.

The class form, when you need state:

python
class Connection:
    def __enter__(self):
        self.conn = connect()
        return self.conn            # what `as` binds

    def __exit__(self, exc_type, exc, tb):
        self.conn.close()
        return False                # False: do not suppress the exception

Returning True from __exit__ swallows the exception. Almost always wrong, and worth knowing so you never do it by accident.

Exercise#

python
from pathlib import Path

# 1. Write "report.txt" containing three lines, using pathlib.
# 2. Read it back line by line and print each with its 1-based line number.
# 3. Write a `counted(label)` context manager that prints how many lines
#    were written inside the block.
# Expected output:
#   1: alpha
#   2: beta
#   3: gamma

Common questions#

Do I still need with if I call close() myself?#

Yes. An exception between open() and close() skips your close() entirely, leaking the descriptor. with closes on every exit path, which is why it is the idiom rather than a convenience.

pathlib or os.path?#

pathlib for new code — the / operator, read_text, rglob and the property accessors are clearer than string manipulation. os.path remains everywhere in existing code and in some library APIs, and the two interoperate fine.

Why does my file have wrong characters on another machine?#

You almost certainly omitted encoding="utf-8", so Python used the platform default, which differs between systems. Always pass it explicitly for text files.

Get the Python agent pack

A battle-tested AGENTS.md, the review checklist, and the failure-mode cheat sheet for Python. One email, then occasional updates when the tooling shifts. No course pitch.

Unsubscribe in one click. We never sell the list. Or just take the AGENTS.md now — no email needed.