Breaking Out of an Async Generator Triggers aclose(): Code After the Last Yield Never Runs

A Python async generator pauses at yield and continues only when the consumer asks for the next value. Once the consumer breaks or drops the generator, asyncio schedules aclose(), which throws GeneratorExit into the suspended yield: statements after the yield don’t run, a finally does. Cleanup placed after the last yield therefore only runs if the consumer reads the stream to the end. Autarkia lost conversation history to this, and the same pattern was fixed once and then put back.

Autarkia’s main chat is an async generator: it pulls tokens from the model, yields events as it goes, and finishes with {"done": True}, which is what tells the client the message is complete. The turn then has to be written to short-term memory, which is slow, so it runs as a background task.

The trouble was the order of those two things. It was changed three times, each time for a reason.

01Three moves

• April 28: the save was an await placed before yield done. The first memory write loads an embedding model and initializes a vector index, about four seconds, and the done signal waited behind it; on the client it looked like a reply that was generated but never shown. The fix moved yield done first and turned the save into create_task.
• April 29: from then on every message was like a new conversation; no history was being saved. The fix moved create_task back before yield done. Since create_task does not block, the April 28 problem stayed fixed. A comment went in next to it: must be scheduled before yield done.
• August 18: during a streaming rework, starting the write before completion seemed wrong; the result is only final once done goes out. So it moved after yield done again. The comment was still sitting right there. Two days later the same symptom came back.

The August version was roughly:

yield {"done": True, "content": current_response_content}
asyncio.create_task(agent._save_conversation_history(**pending_save))

02The generator is parked on the yield

Those two lines read as tell the client it’s done, then save in the background. But an async generator stops at yield and only continues when the consumer asks for the next value. done is the last event, and the consumer breaks as soon as it sees it; in Autarkia both the stream orchestrator and the WebSocket adapter do exactly that. Nobody asks for another value.

Once the generator object is dropped, asyncio schedules aclose() for it, which throws GeneratorExit into the suspended yield. Statements after the yield don’t run; a finally does. This runs as is:

import asyncio, gc

async def gen_after():
    yield {"done": True}
    print("after yield")          # break 时不会打印

async def gen_finally():
    try:
        yield {"done": True}
    finally:
        print("finally")          # break 时也会打印

async def main():
    for g in (gen_after, gen_finally):
        async for ev in g():
            if ev.get("done"):
                break
        gc.collect()
        await asyncio.sleep(0.01)

asyncio.run(main())   # 只打印 finally

If the consumer reads to the end without breaking, the code after yield does run. That is part of why it’s hard to catch.

03Why it slipped through twice

• Whoever writes the generator can’t see the consumer. Inside the generator’s own file, the natural assumption is that events get read to the end, and in that case the code is correct. The break lives in two other files.
• Nothing errors. The GeneratorExit from aclose() is the normal shutdown path, and it leaves nothing in the logs.
• The symptom was the AI forgetting the previous message, which points you at memory retrieval or the model. Proactive messages go through a different path that writes directly and were unaffected, so the history wasn’t completely empty; it looked more like a retrieval problem.
• The April comment said must but not why. The August change had a more specific, more correct-sounding reason, the comment couldn’t stop it, and nothing in the code would fail because two lines moved.

04The fix

On August 20 the save moved into a finally around yield done. The arguments are prepared before the yield, and the write is scheduled in the finally, which runs whether the consumer reads to the end or breaks. That also gives the August rework what it wanted: the write happens after the done signal.

try:
    yield {"done": True, "content": current_response_content}
finally:
    if pending_save:
        spawn_bg_task(agent._save_conversation_history(**pending_save))

The comment changed too, to state the reason: the consumer breaks on done, the generator gets aclosed, and bare code after the yield won’t run. A verification script was added covering three ways of consuming the stream: reading to the end, breaking directly, and breaking through two layers of forwarding. It was run against the old code first to confirm the break cases fail.

One loose end turned up recently: the script is named verify_history_save_not_lost.py and lives under tests/scripts/, while pytest only collects test_*.py, so CI has never run it. A bug that has already regressed once is guarded by a test CI doesn’t run. I plan to rewrite it as a proper test under tests/unit/.