TAG
Measurement
Everything filed under this tag.
4 entries01Writing
The Message You Edited Is Still in Memory2026-11-01After a conversation forks, filtering raw messages by branch is easy, but summaries and facts still remember the abandoned line. Feeding Mem0 only the current branch’s messages lowered its leak rate from 60.8% to just 50.0%.#Memory#Evaluation#Measurement#LLMThe Synthetic Set Picked C. Forty-Four Real Questions Picked B.2026-10-27A synthetic evaluation set gave a clean answer to what granularity memory should be stored at. Real questions from real chats gave a different one — and along the way turned up three measurement problems.#Memory#Evaluation#Measurement#LLMSample Age, Catch-up Sampling, and Measurement Validity2026-10-22In periodic sampling the probe eats into its own window. A 756 ms interval means the curve describes the probe, not the GPU.#Measurement#GPU#SystemsA Failed Batch Stays Failed2026-10-22A run whose failures.jsonl is non-empty can never become a passing record. Fix it and rerun under a new run id; do not fill the gap.#Measurement#Data#Systems