Why ClickHouse and Agents Are a Great Combo

At exe, we collect a lot of telemetry about how our system performs. We use Grafana, Prometheus, and we have all sorts of graphs and alerts. This is great, but it doesn’t say much about what’s happening on the financial side of the business—the churn rate, ARR. Those are things that don’t typically concern engineers.

The obvious answer to this is Stripe Sigma. It’s a solid tool. We used it when I was at 1Password. It’s also expensive and incomplete. Sigma can only see what happens inside Stripe, whereas we bill through the app stores and cloud marketplaces too. Also, a lot of our telemetry is built on ClickHouse. 

Vidrik, who runs accounting, needed three types of reports. Monthly revenue broken down by plan, usage overages, and token pass-throughs (customers buy tokens, we pay the LLM provider).

To solve this, I built a tool called exe-finops. You type a prompt and get a report. The way it works is pretty straightforward.

Exe-finops is built on top of that. It pulls the events out of ClickHouse and puts them into its own SQLite database. Provider webhooks have a lot of information, but if the report needs more (for example, the full customer object) it goes back to Stripe’s API and gets it.

Historically, getting to this point required a real pipeline. Someone built the ETL, Databricks was involved, and Snowflake. There’s all this stuff, in other words, that needed to be built. Exe-finops doesn’t do any of that. It has the events and a reporting layer on top. All three of the reports Vidrik wanted are now prompts.

Exe-finops also supports custom reports, which is useful when you have a question you want answered, like monthly growth, or which days are busiest for billing transactions (I looked a few days ago and it’s Wednesdays and Sundays, no idea why).

I built exe-finops for these reports, but the firehose is also pretty useful. The part of billing that eats the most time is reconstructing what happened from a provider’s dashboard.

Fundamentally, this has made my job easier. When a support question comes in or I need to see what our financial data is doing, I don’t need to fumble around in the Stripe UI or CLI to try and reconstruct it.