Why your observability stack is making your agents stupid (and ours isn't anymore)
2026-11-06 –, Lotus Stage 3

Your observability stack was built for eyeballs, not agents. We wired ours into an LLM the obvious way, with one MCP tool per datasource, and watched context rot make our agents forget their own questions. This talk covers how we rebuilt around Panda, a CLI and MCP server backed by a sandboxed code runtime, and why our agents got smarter, cheaper, and safer.


The 2022-era observability setup is friction for humans and poison for agents: Grafana in front of ClickHouse, Prometheus, and Loki, three query languages, and auth in a browser. We did the obvious integration first, building one MCP tool per datasource. It failed instructively: a single query would dump ~7000 rows of JSON into the context window, rotting the agent's context until it forgot its original task.

We'll walk through the rebuild: Panda, a CLI and MCP server sharing one backend, with a sandboxed Python runtime where raw rows live and die and only summaries return to the model. Humans and agents use the same surface. We'll cover the zero-credential sandbox that makes LLM code against prod safe, and the semantic search that solves discovery. Any core dev can use it today via panda init.

You'll leave with a pattern to steal: give agents a computer, not a pile of tools.

DevOps engineer at ethPandaOps, the Ethereum Foundation team that builds and runs the devnets behind every network upgrade