Review & Verify Updated 2026-09 8 min read View as Markdown

The performance traps in generated Python

Generated Python is usually correct and frequently slow, in a small number of recognisable ways. None of them show up in a test.

A test suite tells you whether code is correct. It says nothing about whether it will still work at a thousand times the row count, and generated code is systematically biased towards the shape that is clearest at small scale rather than the shape that survives.

This is not a criticism of the models — clarity is usually the right default, and it is what the training data rewards. It just means performance review is now a distinct pass, and it is worth knowing what to look for.

1. N+1 queries#

The most expensive item on this list in practice, and the most consistently generated.

orders = await db.fetch_all("SELECT * FROM orders WHERE user_id = :u", {"u": uid})
for order in orders:
    order.items = await db.fetch_all(                      # one query per order
        "SELECT * FROM items WHERE order_id = :o", {"o": order.id}
    )

Correct. Passes every test, because the test has three orders. In production with a hundred orders per user, that is 101 round trips.

Look for: any await or .query( inside a for loop.

Catch it with: a test that asserts query count, which is the only reliable defence.

async def test_order_list_is_two_queries(db_spy, client, user):
    await client.get("/orders", headers=user.auth)
    assert db_spy.count <= 2

With SQLAlchemy, selectinload/joinedload fixes it; with raw SQL, one query with WHERE order_id = ANY(:ids). Either way the test is what stops it coming back.

2. Quadratic membership tests#

for item in new_items:              # 10,000
    if item.sku in existing_skus:   # a list of 10,000
        ...

in on a list is O(n). Fine at ten, 100 million comparisons at ten thousand. set(existing_skus) once, outside the loop, and it is instant.

Look for: in against a list or a .keys() inside a loop; repeated .index(); list.remove() in a loop; building a list with += inside a loop.

This one is genuinely common because at tutorial scale it is invisible.

3. iterrows() and friends#

for idx, row in df.iterrows():
    df.at[idx, "total"] = row["qty"] * row["price"]

Generated pandas reaches for row iteration because that is what most pandas in the training data does. It is typically hundreds to thousands of times slower than the vectorised form:

df["total"] = df["qty"] * df["price"]

Look for: iterrows, itertuples in a hot path, apply(axis=1), df.append in a loop (which is quadratic — it copies the frame every time).

4. Loading everything into memory#

rows = cursor.fetchall()                    # the whole table
data = json.loads(open(path).read())        # the whole file
lines = open(path).readlines()              # all of it
df = pd.read_csv(huge)                      # no chunksize, no dtype

Works on the 10MB sample. Gets OOM-killed on the real 8GB file, at 3am, in a job with no retry.

Correct: server-side cursors, ijson or a streaming parser, iterate the file object directly, chunksize= on read_csv.

Look for: fetchall, .read(), readlines(), any read_csv without chunksize or dtype on a path that could be large. Also: generated code loves building a full list before returning it, where a generator would do.

5. Blocking calls in async code#

Covered in the failure-mode catalogue as a correctness bug, but its real cost is performance, and it is the most damaging item here because it is invisible until you are under concurrent load.

async def handler(request):
    data = requests.get(url).json()         # blocks the entire event loop
    result = expensive_cpu_thing(data)      # so does this

One blocking call in one handler serialises the whole process. Throughput collapses and every trace looks fine individually.

Catch it with: ruff's ASYNC rules for the obvious cases, and asyncio debug mode (PYTHONASYNCIODEBUG=1) in tests, which warns on any coroutine that blocks for more than 100ms.

6. Recomputing what does not change#

def price(items):
    rates = load_exchange_rates()           # network call, per invocation
    config = json.load(open("config.json")) # file read, per invocation
    return sum(...)

Generated functions are self-contained by preference — it makes them easier to read and easier to test — which means expensive setup migrates inside them. Hoist it, cache it (functools.lru_cache, or module-level for genuinely static data), or pass it in.

7. Regex compiled in a loop#

re.compile inside a hot loop, or re.sub with a complex pattern per row. Python caches compiled patterns, so this is usually a small effect — but catastrophic backtracking on a generated regex is not small, and generated regexes are frequently more nested than they need to be.

Worth checking: any generated regex with nested quantifiers ((a+)+, (\s*\w+)*) applied to input you do not control. That is a denial-of-service vector, not just slowness.

How to actually find these#

Reading for performance is unreliable. Measure instead, and make it cheap enough that you do it.

# where did the time go?
uv add --dev py-spy
uv run py-spy record -o profile.svg -- python -m yourapp.job

# what did that endpoint do?
uv run pytest --durations=10          # slowest tests are usually slowest code

Two habits worth more than any checklist:

1. Test with realistic data volumes at least once. A fixture with 10,000 rows instead of 3 finds most of this page automatically, and you only need it for the handful of paths that matter.
2. Assert on query counts for anything that talks to a database. It is the only test that reliably catches N+1, and N+1 is the expensive one.

Common questions#

Should I ask the agent to optimise the code?#

Not upfront. Premature optimisation is as bad here as anywhere, and generated "optimised" code is often less clear for no measurable gain. Ask for correct and clear, then profile, then optimise the one thing that showed up. The exception is N+1 queries — those are worth catching structurally rather than by profiling, because the fix is architectural.

Do these problems show up in tests?#

Almost never, and that is the whole issue: tests run with three rows. A query-count assertion and one fixture with realistic volume are the two additions that make the test suite able to see this class of bug at all.

Is the profiling worth it for a small app?#

Yes, once, when something feels slow — py-spy needs no code changes and takes about a minute. What is not worth it is routine profiling of code nobody has complained about.

What about the GIL and free-threaded Python?#

Relevant for CPU-bound work, and increasingly so as free-threaded builds mature, but it is rarely the problem in the code this page is about. The typical generated bottleneck is I/O in a loop, not contention — fix the loop first, and only then ask whether threads would help.

Disclosure: some links on this page are affiliate links. If you buy something through one, we earn a commission at no extra cost to you. We only list tools we would tell a friend to use, and we say so when we have not used something ourselves. This is how the site stays free and ad-light.