Review & Verify Updated 2026-09 16 min read View as Markdown

The Python mistakes language models actually make

A working catalogue of the bugs that show up over and over in generated Python — what each one looks like, why models produce it, and the check that catches it.

Generated Python fails in patterns, not at random. The patterns come from the training data: a model has read millions of lines of Python, most of it written before 2023, much of it tutorial code that was never load-bearing. When it is uncertain, it reaches for the most common thing it has seen — and the most common thing is frequently the thing that was fine in a blog post and is not fine in your service.

This is the list we keep coming back to. Each entry has the shape of the bug, why it appears, and the mechanical check that catches it so you do not have to rely on noticing.

Correctness#

1. Mutable default arguments#

def add_item(item, basket=[]):        # every call shares one list
    basket.append(item)
    return basket

The classic. It appears because it is enormously represented in the training data — in both directions, as an example of the bug and as real code. Models reproduce the shape.

Correct: def add_item(item, basket: list | None = None): basket = [] if basket is None else basket

Catch it with: ruff rule B006 (flake8-bugbear). Non-negotiable in select.

2. Naive datetimes#

created = datetime.utcnow()           # deprecated, and tz-naive
if created < deadline:                # comparing naive to aware -> TypeError
    ...

datetime.utcnow() has been deprecated since 3.12 but is overwhelmingly present in training data. The resulting naive datetime then flows into comparisons, database columns and serialisation, and produces bugs that only appear across a DST boundary or in a different deployment region.

Correct: datetime.now(timezone.utc) — and pick one rule (everything aware, UTC) and put it in AGENTS.md.

Catch it with: ruff rule DTZ (flake8-datetimez). Turn on the whole family.

3. Floats for money#

total = 0.1 + 0.2                     # 0.30000000000000004
price = round(qty * unit_price, 2)    # rounds half-to-even, surprising in finance

Models default to float because most numeric Python in training data is scientific, where float is right. In billing it is not.

Correct: Decimal, constructed from strings, with an explicit quantize and rounding mode at the boundary.

Catch it with: a type. Make your money type Decimal in the domain layer and let mypy reject the float at the door. No linter finds this one for you.

4. Late binding in loops#

handlers = [lambda: print(i) for i in range(3)]
[h() for h in handlers]               # 2, 2, 2 — not 0, 1, 2

Shows up most in generated callback registration, retry wrappers and click handlers.

Correct: lambda i=i: print(i), or functools.partial.

Catch it with: ruff rule B023 (function definition does not bind loop variable).

5. Blocking calls inside async def#

async def fetch_user(uid: str):
    r = requests.get(f"{API}/users/{uid}")   # blocks the whole event loop
    time.sleep(0.2)                          # so does this
    return r.json()

The most damaging item on this list, because it does not fail — it just quietly serialises your entire service and shows up as latency under load, weeks later.

Models produce it because requests is far more represented in training data than httpx, and because the code reads correctly.

Correct: httpx.AsyncClient, asyncio.sleep, aiofiles, and asyncio.to_thread() for anything genuinely blocking.

Catch it with: ruff rules ASYNC (flake8-async), plus blockbuster or asyncio debug mode in tests. Add ASYNC to select today if you run any async code.

6. Broad exception handling#

try:
    result = risky()
except Exception:
    result = None                     # the incident report starts here

Appears whenever a prompt contains the phrase "handle errors gracefully". The model interprets graceful as silent.

Correct: catch the specific exception you can do something about; let the rest propagate. If you must catch broadly at a boundary, log with exc_info=True and re-raise or return a typed failure.

Catch it with: ruff rules BLE001 (blind except) and S110 (try-except-pass).

7. assert for runtime validation#

def withdraw(account, amount):
    assert amount > 0, "amount must be positive"   # vanishes under python -O

Correct: if amount <= 0: raise ValueError(...).

Catch it with: ruff rule S101, with a per-file ignore for tests/.

8. Mutating a collection while iterating it#

for user in users:
    if user.inactive:
        users.remove(user)            # skips elements, silently

Correct: build a new list, or iterate a copy (for user in list(users)).

Catch it with: ruff rule B909 where available; otherwise a review habit. Property tests catch the resulting off-by-one behaviour reliably.

9. Dict and set ordering assumptions#

Dicts preserve insertion order since 3.7. Sets do not, and set iteration order varies between runs when PYTHONHASHSEED is randomised. Generated code frequently builds a set, iterates it, and produces output that is stable on the developer's machine and flaky in CI.

Catch it with: run your test suite twice with different PYTHONHASHSEED values in CI. It is one environment variable and it finds real bugs.

10. is versus ==#

if status is "active":                # works for interned strings, then stops working
if count is 0:                        # same

Catch it with: ruff rule F632. This one is well covered — just make sure F is in select.

Data and pandas#

11. Chained assignment#

df[df.score > 0.9]["flag"] = True     # writes to a copy; original unchanged

Silent no-op in older pandas, an error under copy-on-write in pandas 3. Either way, the generated analysis is wrong.

Correct: df.loc[df.score > 0.9, "flag"] = True.

Catch it with: run with copy-on-write enabled (pd.options.mode.copy_on_write = True) so it raises rather than warns.

12. inplace=True#

Still ubiquitous in generated pandas, largely deprecated, rarely faster, and it defeats method chaining. Prefer reassignment.

13. iterrows() for anything#

Generated pandas reaches for row loops because that is what tutorial pandas does. On a million rows it is thousands of times slower than the vectorised form. This one is not a correctness bug — it is a "why is the job taking six hours" bug.

Security#

The full treatment is in the security review checklist; these are the ones that recur most in Python specifically.

14. subprocess with shell=True and an f-string#

subprocess.run(f"git log --author={author}", shell=True)   # command injection

Correct: a list of arguments, shell=False (the default).

Catch it with: ruff rule S602/S605 (bandit rules via flake8-bandit — enable the S family).

15. yaml.load without a loader, pickle on untrusted input#

Both execute arbitrary code by design. Both appear in generated config-loading and caching code.

Correct: yaml.safe_load; json instead of pickle for anything crossing a trust boundary.

Catch it with: ruff rules S506 and S301.

16. random for tokens#

token = "".join(random.choices(string.ascii_letters, k=32))   # predictable

Correct: secrets.token_urlsafe(32).

Catch it with: ruff rule S311.

17. String-built SQL#

cur.execute(f"SELECT * FROM users WHERE email = '{email}'")

Still generated, especially when the surrounding code does not use an ORM. Catch it with: ruff rule S608, and a hard convention of parameterised queries.

Housekeeping that becomes correctness#

18. open() without an encoding#

open(path) uses the platform default, which is UTF-8 on modern Linux and macOS and was historically not on Windows. Generated file-handling code omits it constantly. Catch it with: ruff rule PLW1514 / W1514.

19. Shadowing stdlib module names#

Files called types.py, logging.py, secrets.py, email.py next to code that imports the stdlib module of the same name. The failure is an import error far from the cause. Catch it with: ruff rule A005.

20. Silent integer division changes#

// versus / in ported or translated code. Not a linter catch — a test catch. Anywhere you see a division in generated numeric code, write the boundary test.

Turning most of this on#

Nearly two thirds of the list above is enforceable with one config block:

[tool.ruff.lint]
select = [
  "E", "F",        # pycodestyle, pyflakes
  "B",             # bugbear    -> mutable defaults, loop binding
  "S",             # bandit     -> shell=True, yaml.load, random, SQL
  "DTZ",           # datetimez  -> naive datetimes
  "ASYNC",         # async      -> blocking calls in coroutines
  "BLE",           # blind except
  "A",             # builtins/stdlib shadowing
  "PL",            # pylint subset -> encoding, misc
  "SIM", "UP", "I", "RUF",
]
ignore = ["E501"]

[tool.ruff.lint.per-file-ignores]
"tests/**" = ["S101"]

uv add --dev ruff
uv run ruff check --statistics .     # see what you are already shipping

Common questions#

Are these bugs unique to AI-generated code?#

No — every one of them predates language models and appears in human code too. What changed is the rate and the distribution. Models produce the most-represented pattern rather than the most-appropriate one, so these particular mistakes arrive far more consistently, and they arrive in volume, in code that looks confident and well-formatted.

Does a better model make this list shorter?#

Somewhat, and unevenly. The obvious ones — is versus ==, mutable defaults — are largely gone from frontier models. The subtle ones survive, because they are subtle: blocking calls in async code and float money still appear regularly, since both produce code that reads correctly and passes a naive test.

Is a linter really enough?#

For roughly two thirds of this list, yes, and that is the point — those items should stop consuming your attention entirely. The remaining third (money types, division semantics, iteration-order assumptions, whether the code solves the right problem) needs tests and human judgement. Spend your review time there.

What is the single highest-value thing to turn on?#

If you write async Python, the ASYNC rule family, because blocking-call-in-coroutine is the one failure here that is both common and effectively invisible until you are under load. If you do not, B and S together give you the largest reduction in real defects per minute spent.

Disclosure: some links on this page are affiliate links. If you buy something through one, we earn a commission at no extra cost to you. We only list tools we would tell a friend to use, and we say so when we have not used something ourselves. This is how the site stays free and ad-light.