Thinking Like a Data Architect

Last year I wrote about thinking like a data engineer and shared my journey in data engineering. After building large-scale data infrastructure for various leading companies, I wrote about what it means to think like a data architect. People don’t break the system; a poorly designed system breaks people. Your data architecture decides where the work lands. It decides who has to know which table is current, who has to explain why two dashboards disagree, who pays for a migration, and who decides at 2 a.m. whether an incident is really over. So when teams struggle with data, I don’t start by judging someone’s decision. I start by asking what the system made easy, what it made hard, and what it left them to figure out on their own. Building observability at Slack shaped this habit.* It taught me a simple test: after someone sees the signal, what can they actually do? An alert can identify a problem accurately and still leave its recipient unsure who owns it or how to respond. The system detected the problem. It did not help anyone solve it. The same test applies well beyond alerts. In the comments on my previous article, Thinking Like a Data Engineer, Ana summed up the tradeoff in one line: “One person’s walk signal is another person’s wait time.” Every design choice that makes one team’s work easier can move the difficulty into someone else’s day. That difficulty accumulates in five recurring places: discovery, publication, metric definitions, migration, and recovery. In each one, the fix is the same: move it. Take work that every consumer does repeatedly and alone, and turn it into a promise the system keeps once. If you are a data architect or head of data, this is your job. You just don’t feel where the work lands every day. You decide where it lands. Discovery: Stop Making a Colleague the Interface A team builds a transformation on a table that it thought was the source of truth. Weeks later, someone explains that it chose the wrong table. There was a newer dataset, a different owner, or business logic the team should have reused. It is easy to ask why the team did not check. The better question is whether we gave it a dependable way to check. A catalog helps people find assets, and lineage shows where they came from. Neither tells a consumer which asset to build on. That takes a supported entry point: a data product with an accountable owner, an explicit contract, a stated purpose, and visible freshness. Certification needs a scope, too. A dataset certified for exploration may lack the completeness guarantees a customer-facing report requires. “Certified” without “for what” is only a badge. If choosing the right dataset depends on knowing the right colleague, we have made that colleague part of the interface. It is an interface with no documentation and no SLA, and it goes on vacation. A good entry point lets the next engineer make a sound decision without first learning the organization’s unwritten history. Publication: Declare Which Version Consumers Can Depend On At 6 a.m. on Wednesday, one pipeline takes a snapshot of Tuesday’s orders: 41,200. A second pipeline keeps counting as late events arrive, such as orders from mobile apps that were offline overnight. By noon, it shows 43,050 for Tuesday. Both numbers are labeled “Tuesday.” Both are correct. Then an analyst puts the first number in the morning report, a manager opens the dashboard after lunch, and someone has to explain why they don’t match. My engineering instinct is to explain the gap. But an explanation does not answer the question the consumer actually has: which number should I use, and will it change again? Multiple publications can serve legitimate needs. But they cannot travel without their contract. Every publication should declare its purpose, its snapshot cutoff, whether late arrivals and corrections will revise it, and a version identifier a consumer can cite. Without those, two correct numbers become one argument. Reproducibility is part of the same promise. Time travel helps only as long as the history is retained, so data and logic retention must match how long consumers may need to reproduce a result. Backfills, replays, and contract changes need a publication process that makes a correction recognizable as a correction. An architect can understand exactly why two numbers differ while the person presenting them still cannot say which one applies. Settle the promise before asking someone else to defend the result. Metrics: Version the Meaning, Not Just the Schema Sometimes the numbers disagree even after the timelines match. The product counts conversion events. A customer report counts customers who converted. Finance applies its own attribution rule. Each is reasonable in its own context. The trouble starts when we present them as the same measure. Steven Strogatz’s Sync describes how fireflies flashing in unison and the pacemaker cells of a heart fall into rhythm with no conductor. Organizations do not work that way. Teams do not converge on shared meaning by proximity; left alone, their definitions drift apart. In a data platform, the connections that keep teams in rhythm are shared definitions and explicit promises, and we have to build them on purpose. A shared metric needs a versioned definition and an accountable owner. A glossary entry is not enough: it records intent but cannot verify an implementation. Teams should either reuse one computation or prove their implementations agree by passing the same shared test cases. Matching inputs, definitions, and versions should produce matching results, and a test suite shows you that they do. Different business purposes can still justify different definitions. When they do, give them different names and document how they relate, so consumers don’t mistake one for the other. This is the encouraging part. Once two teams agree on what they exchange, a third team can reuse that agreement instead of negotiating it again. Migration: Put the Cost of Change in the Plan A platform team proposes retiring an older dataset. The replacement has clearer ownership and a better contract. Running both costs money and doubles the places where things can break. Then an engineer from a consuming team asks: “Which of our commitments should we move to make time for this migration?” That question deserves a better answer than another walkthrough of the target architecture. The platform team has maintenance it needs to stop carrying. The consuming team has promises it needs to keep. Each is asking the other to absorb work its own plan does not account for. I can describe the destination in detail and still leave almost the entire cost of getting there to someone else. Stewart Brand’s How Buildings Learn offers a better model. Drawing on architect Frank Duffy’s work, Brand describes a building as layers that change at different rates. The structure lasts for decades, while services and interior layouts change far more often. A data platform has layers too. An implementation can change freely behind a stable contract. A change in meaning cannot. Extending an attribution window from 7 to 30 days changes results while leaving every column name and type intact. Schema compatibility checks will pass. The contract has still changed, and every consumer needs a deliberate transition. So make the cost of adaptation visible and planned:

• Affected consumers: use lineage to identify every one of them before announcing the change.
• Coexistence window: agree on how long both versions run, with a firm end date.
• Migration ownership: give the work an owner and a line item in the consuming team’s delivery plan, not only the platform team’s.
• Verification: give QA representative data and an environment to test behavior, including the differences expected between versions. Recovery: Design the Way Back Before You Need It A defect duplicates some events—an alert fires. We fix the transformation, rebuild the affected tables, and the pipeline is green again. The wrong result may still sit in a cache, a derived aggregate, or a report someone downloaded yesterday. A customer may already have acted on it. A healthy pipeline is only part of the answer. This is where the observability test returns: what can the person receiving the signal actually do? Alerts should state business impact and reach someone who can act. Acknowledgment and escalation make it visible whether anyone is responding. Noisy alerts steal the attention you need for a serious failure. Past the alert, the responder needs a path through repair:
• Blast radius: lineage identifies the affected outputs and consumers.
• Safe rebuilds: replays and backfills are idempotent, so a retry never introduces new duplicates.
• Recognizable corrections: restated outputs carry a new version identifier and a correction notice.
• Downstream closure: someone verifies derived results and tells consumers when an exported report has been superseded. All of this depends on retention. Reproducing a result can require the historical inputs, code, metric definitions, and reference data from that moment. Agree on a recovery window and retain what that promise requires. Otherwise, the responder reconstructs those decisions under pressure, while another team waits to hear whether the incident is really over. Who Pays for All This Each mechanism above adds work for producers and platform teams: scoped certification, versioned publications, tested metric definitions, coexistence windows, and retained history. That is not a side effect. It is the point. Without these mechanisms, the same work still happens. It happens repeatedly, informally, and under pressure, done by every consumer who has to trace a number, reconcile two dashboards, or decide whether an incident is closed. Architecture cannot make that work disappear. It can move it to the team that can do it once, do it well, and keep doing it. Making that call is the data leader’s job. No single consumer team will ask for this work, because each one sees only its own share of the cost. Only someone who sees the whole platform can see the total. That is the walk signal, applied deliberately. Some wait time is worth introducing when it spares everyone else at the intersection. Building for the People Who Come Next Go back to the engineer in that design review. A diagram showed where the data should go. Their question showed where the work would land. Following a decision far enough to see who has to live with it is the habit I am still building. A workaround usually points to an unmet need. Repeated confusion usually points to an agreement we left unfinished. If you lead a data platform, start with one place where people repeatedly struggle. Ask what they are trying to accomplish and what they currently have to remember or work around. Then change one thing so the next attempt is easier: a scoped certification, a version identifier on a publication, a shared test case for a metric. An architecture’s measure isn’t how its diagram looks. It is whether the next person can learn, change things safely, and recover when something goes wrong. All rights reserved, Dewpeche Pvt Ltd, India. I have provided links for informational purposes and do not suggest endorsement. All views expressed in this newsletter are my own and do not represent current, former, or future employers’ opinions.