TAG
Observability
Everything filed under this tag.
3 entries01Writing
LLM Request Observability: Logs Count Messages, and the Native Tools Schema Shows Up Nowhere2026-11-07With native function calling, tool definitions go in the tools parameter, not in messages. A log that measures message length can’t see them, and streaming responses omit usage by default. Autarkia sent 82 tools and 47,898 characters of schema every turn while the prompt log showed about 2,200 characters; the API counted 21k–23k tokens.#LLM#Python#Observability#AutarkiaWhere the Prefix-Cache Hits Went2026-10-27The hit rate dropped, and the total alone couldn’t say why. Looked at in 128-token blocks, the losses came from one section placed in the wrong spot, and from two providers with completely different caching rules.#LLM#Cost#ObservabilityOne pop() in a subclass payload builder: max_tokens deleted before sending, invisible to callers, logs and tests2026-10-21The parent class builds the request body, the subclass pops a field before returning it, and whatever the caller passes stops mattering without any error. Autarkia’s MiniMax client has deleted every caller’s max_tokens since the first commit, leaving a reasoning model with no output ceiling, and it was mentioned three times in my notes without being changed.#Python#LLM#Observability