The Four Kinds of Cloud Architecture Debt

• Tue, Apr 19, 2022
• 9-minute read

Cloud architecture debt doesn’t announce itself. It accumulates in the gaps between the decisions someone made and the documentation they didn’t leave behind: the IAM policy nobody can justify, the VPC peering connection from a project that shipped two years ago, the data classification spreadsheet that was current when someone made it. None of this comes from incompetence. Each decision made sense when somebody made it. Each role solved a problem somebody had on a Tuesday. Each peering connection unblocked a project that needed to ship.

What organizations pile up, over years of this, is cloud architecture debt. Like financial debt, it costs more to ignore than to retire, but the bill never lands in a form anyone can attribute to the buildup. Unlike financial debt, nobody signed for it, and the principal balance doesn’t show up in any single report.

List the items individually and you’ll paralyze yourself. Most are noise on their own. The categories aren’t. Cloud architecture debt comes in four shapes, and once you can read the shape of what you’re carrying, the triage gets much easier.

Figure 1. The four kinds grouped by visibility. Governance and operational debt show up in incident reviews and audits. Knowledge and optionality debt stay invisible until you need the option you no longer have, and by then the cleanup cost has compounded.

Governance debt

Governance debt is the category where the cost comes from consensus you didn’t build at the time. The decisions that should have been written down and weren’t. The policies that lived in someone’s head and disappeared with the reorg. The “we’ll do it properly later” account topology that turned out to be the only topology they had.

The right moment to make the decision was three years ago. The only moment you have now is this one, and the people who’d have built the consensus have moved on. At a research organization I worked with, one approaching a compliance audit with several access-control gaps still open, the cleanup took six months. Two production services were running in the wrong account. Neither had a current owner.

Code rarely fixes governance debt. Cross-account access “works” but no one can explain how. IAM roles run without owners. Compliance posture depends on tribal knowledge that walked out the door. Every architecture review surfaces the same “we should really document that.” A forum that can hear and resolve disagreement, with named owners and written records, fixes it. None of those exist by accident.

Operational debt

Operational debt fires expensively at moments: every incident, every audit, every onboarding. It stays invisible the rest of the time. That’s what makes it persistent. The cost shows up to one engineer at a time, never to leadership in a form that gets budget.

A SaaS company I worked with, one that had grown faster than its deployment tooling, had years-old custom deployment scripts that only a handful of engineers fully understood. Onboarding to the on-call rotation took longer than onboarding to the company. Senior engineers quietly rotated new hires off, because new hires escalated to one of the three within an hour. Runbooks pointed to commands that didn’t exist anymore. Alerts fired for conditions nobody had checked the meaning of since the team that wrote them changed shape. Above the team level, none of it was visible. Each individual incident resolved cleanly enough.

Naming operational debt is easy. Funding the cleanup is the hard part, and this is the one I find hardest to call cleanly from the inside: the cost of operational debt is real but distributed, one engineer at a time, one incident at a time, and the people who feel it most aren’t usually in the room when budget gets allocated. The conversations that get funding are the ones where that distributed cost becomes legible to people who aren’t on call.

Knowledge debt

Knowledge debt compounds in a way the others don’t. Every new engineer onboards into a slightly more opaque system. Every retiring senior takes a piece of the map with them. From the inside, the slope is hard to feel, which means time-to-productive for new hires creeps up year over year, until one day a third of the team spends its time maintaining things nobody can fully explain.

Call it Chesterton’s fence at scale. Every running service is there for a reason, and the reasons are gone. A platform team I advised, one that had seen most of its founding engineers depart over two years, had lost the briefing on most early decisions. Several internal services sat there nobody would deprecate, “in case something depends on them.” The dependency graph was knowable, in principle, but mapping it had grown more expensive than leaving the services running.

Knowledge debt leaves marks. Once you look for them, they’re everywhere: “I think Sarah set that up before she left” cited as an architectural fact, README files three or more years stale, the Terraform module nobody touches because nobody can predict what would break.

Optionality debt

Optionality debt stays invisible until you need the option you don’t have. By then, paying it down costs more than the original decision saved. Lock-in happens by accident. The decisions that bind you to a vendor, a region, a runtime, or a license model rarely get made explicitly. The monorepo started as one service. The cloud-specific feature got used because it was easy, and now it’s wired through everything.

A SaaS company I worked with, one that had made all the right technical calls for their scale at the time, ran fine on a single cloud’s proprietary database, even at near-total adoption across services. Then their largest customers started asking for cross-cloud deployment. The migration estimate came in at multiple years. A product roadmap item went on hold waiting for it. The team that built the original architecture was right at the time. Nobody revisited the decision as the world changed around it.

The signs are quiet: “We can’t move off X because Y,” where Y is something nobody noticed had been written into the architecture. Vendor renewals where the negotiation isn’t a negotiation. Regional concentration that wasn’t a strategic choice. None of these announce themselves as debt at the time. They announce themselves the first time you need to act and discover you can’t.

Reading the four shapes together

The four categories aren’t equally expensive to carry, equally visible, or equally tractable. A side-by-side comparison makes the triage more practical.

The table shows why mixed backlogs fail: governance needs an executive sponsor, operational needs a platform team, knowledge needs hiring-and-onboarding investment, and optionality needs a strategy conversation. No single owner can address all four, which is why the items compete poorly when listed together.

How to triage cloud architecture debt

Not every debt is paying interest. Some of it sits dormant. Some is structural, and patches just rearrange the cost. The distinction matters: retiring structural debt costs much more than retiring dormant debt. Pitch the wrong scope to leadership, and the cleanup loses funding before it starts.

The honest triage: pay interest first, refinance second, live with the rest. Debt that costs you now, in incidents, in slow shipping, in compliance scope, earns the most by being retired. Some debt gets cheaper without retiring: better documentation, better automation, better fences around it. Dormant debt that costs nothing and would cost a lot to retire belongs on the balance sheet, not the backlog.

Figure 2. A triage decision tree for individual debt items. Most items resolve to one of four answers. The costliest mistakes happen when teams treat every item as “pay first” and run out of capacity, or treat every item as “live with it” and accumulate quietly until the next audit forces the conversation.

The quarterly review template below gives the triage a repeatable form. The key discipline is requiring a named owner before any item can move to “active”:

# Architecture debt review template
# Use once per quarter per category. Named owner required for each item.

governance:
  - item: "IAM roles with no documented owner"
    status: active          # active | dormant | retiring
    owner: ""               # Required
    review_date: ""
    cost_to_carry: high     # high | medium | low
    cost_to_retire: high

operational:
  - item: "Custom deploy scripts (pre-CI migration)"
    status: active
    owner: "platform-team"
    review_date: "2022-07-01"
    cost_to_carry: medium
    cost_to_retire: medium

knowledge:
  - item: "Undocumented legacy service: svc-report-etl"
    status: dormant
    owner: ""
    review_date: ""
    cost_to_carry: low
    cost_to_retire: high    # mapping + dependency work

optionality:
  - item: "RDS Aurora proprietary features (five services)"
    status: dormant
    owner: "architecture"
    review_date: "2022-10-01"
    cost_to_carry: low
    cost_to_retire: very_high

Resist the temptation to flatten the four into one backlog. Each category needs different sponsorship: governance is an executive conversation, operational a platform-team conversation, knowledge a hiring-and-onboarding conversation, optionality a strategy conversation. Mixed backlogs lose all four because no single person is the right owner for the whole list.

Making cloud architecture debt visible

You’ll never be debt-free. The pattern that holds up is that healthy organizations don’t carry less architectural debt than the rest. They name what they carry and hold a defensible position on each piece. The unhealthy version isn’t the team with more debt. It’s the team whose engineers know the debt exists, leadership doesn’t, and the conversation that would connect them never quite happens.

The healthcare org I mentioned paid down their governance debt over six months. The work that opened up after wasn’t the cleanup. It was the conversations they could finally have with leadership about what to build next. What held them back for years wasn’t the debt itself: it was that the people who could fund the work couldn’t see it. Once they could, the funding was straightforward. The hard part had always been the translation.

The question worth sitting with is this: in your organization, which of the four kinds would leadership fund if it were visible, but can’t currently see at all?

Related Posts

• What a cloud architecture review should produce
• Shaping architecture with SLOs
• The implicit architecture of every modernization decision
• Infrastructure security architecture patterns
• What going containerized committed you to