# Files and Context Managers

> Source: https://learn-python.com/files-and-context-managers/
> Part of Learn Python, free to read.

```python
with open("notes.txt", "r", encoding="utf-8") as f:
    content = f.read()
# the file is closed here — even if an exception was raised
```

The `with` statement is the point. It guarantees the file is closed on every path out of the block, including an exception, which is why you should essentially never call `open()` without it.

:::warn Always pass `encoding`
```python
open("notes.txt")                      # uses the platform default
open("notes.txt", encoding="utf-8")    # explicit, portable
```
Without it, the same code reads different bytes on different machines, and the failure is a `UnicodeDecodeError` in production on text that worked locally. Generated Python omits it constantly. `ruff`'s `PLW1514` rule catches it.
:::

## Modes

| Mode | Meaning |
|---|---|
| `"r"` | read (default); error if the file is missing |
| `"w"` | write; **truncates an existing file** |
| `"a"` | append |
| `"x"` | create; error if it already exists |
| `"rb"` / `"wb"` | binary — no decoding, gives you `bytes` |
| `"r+"` | read and write |

`"x"` is underused: it is the mode that refuses to overwrite, which is what you want for anything you are creating for the first time.

## Reading

```python
with open("data.txt", encoding="utf-8") as f:
    content = f.read()          # the whole file as one string

with open("data.txt", encoding="utf-8") as f:
    for line in f:              # one line at a time — lazy, no size limit
        print(line.rstrip("\n"))
```

**Iterate the file object** rather than calling `.readlines()`. Iteration streams; `readlines()` loads everything into memory, which is fine for a config file and fatal for an 8GB log.

```python
with open("big.log", encoding="utf-8") as f:
    errors = sum(1 for line in f if "ERROR" in line)   # constant memory
```

## Writing

```python
with open("out.txt", "w", encoding="utf-8") as f:
    f.write("first line\n")               # write does NOT add a newline
    f.writelines(f"{n}\n" for n in range(3))
    print("via print", file=f)            # print does add one
```

## pathlib

The modern way to handle paths. Prefer it to `os.path` string juggling:

```python
from pathlib import Path

config = Path("config") / "settings.json"      # / joins, cross-platform
config.exists()
config.suffix                                   # ".json"
config.stem                                     # "settings"
config.parent                                   # Path("config")

text = config.read_text(encoding="utf-8")       # open, read, close
config.write_text(data, encoding="utf-8")
raw = config.read_bytes()

Path("output").mkdir(parents=True, exist_ok=True)

for py in Path("src").rglob("*.py"):            # recursive glob
    print(py)
```

`read_text` and `write_text` handle the open and close for you, which makes `with` unnecessary for small whole-file operations.

## Writing safely

A crash partway through a `"w"` write leaves a truncated file. Write to a temporary file and rename:

```python
import os, tempfile
from pathlib import Path

def write_atomic(path: Path, data: str) -> None:
    fd, tmp = tempfile.mkstemp(dir=path.parent, suffix=".tmp")
    try:
        with os.fdopen(fd, "w", encoding="utf-8") as f:
            f.write(data)
            f.flush()
            os.fsync(f.fileno())      # actually on disk
        os.replace(tmp, path)         # atomic within a filesystem
    except BaseException:
        os.unlink(tmp)
        raise
```

`os.replace` is atomic, so a reader sees either the old file or the new one — never a half-written one.

## What `with` actually does

Any object implementing `__enter__` and `__exit__` works with `with`. It is a general resource-management protocol, not a file feature:

```python
with open("a.txt") as f: ...           # closes the file
with lock: ...                          # releases the lock
with conn.transaction(): ...            # commits or rolls back
with tempfile.TemporaryDirectory() as d: ...   # removes the directory
```

Several at once:

```python
with open("in.txt") as src, open("out.txt", "w") as dst:
    dst.write(src.read())
```

## Writing your own

The decorator form covers almost every case:

```python
from contextlib import contextmanager
import time

@contextmanager
def timed(label: str):
    start = time.perf_counter()
    try:
        yield                                   # the body of the `with` runs here
    finally:
        print(f"{label}: {time.perf_counter() - start:.3f}s")

with timed("import"):
    process_everything()
```

Everything before `yield` is setup, everything after is cleanup, and `finally` guarantees the cleanup runs even when the body raises. That try/finally is the part people forget, and without it the context manager silently stops cleaning up on the exact path where cleanup matters most.

The class form, when you need state:

```python
class Connection:
    def __enter__(self):
        self.conn = connect()
        return self.conn            # what `as` binds

    def __exit__(self, exc_type, exc, tb):
        self.conn.close()
        return False                # False: do not suppress the exception
```

Returning `True` from `__exit__` swallows the exception. Almost always wrong, and worth knowing so you never do it by accident.

## Exercise

```python
from pathlib import Path

# 1. Write "report.txt" containing three lines, using pathlib.
# 2. Read it back line by line and print each with its 1-based line number.
# 3. Write a `counted(label)` context manager that prints how many lines
#    were written inside the block.
# Expected output:
#   1: alpha
#   2: beta
#   3: gamma
```

## Common questions

### Do I still need `with` if I call `close()` myself?

Yes. An exception between `open()` and `close()` skips your `close()` entirely, leaking the descriptor. `with` closes on every exit path, which is why it is the idiom rather than a convenience.

### `pathlib` or `os.path`?

`pathlib` for new code — the `/` operator, `read_text`, `rglob` and the property accessors are clearer than string manipulation. `os.path` remains everywhere in existing code and in some library APIs, and the two interoperate fine.

### Why does my file have wrong characters on another machine?

You almost certainly omitted `encoding="utf-8"`, so Python used the platform default, which differs between systems. Always pass it explicitly for text files.