Green Charts, Leaving Users: When Your Metrics Are Lying to Your Face

There's a particular kind of meeting that happens at a lot of software teams. Everyone gathers around a screen full of charts. The lines are mostly going up and to the right. Someone nods approvingly. Someone else mentions that deployments are at an all-time high. The meeting ends and everyone feels good about the week.

Meanwhile, three of your best users didn't come back.

This is metrics theater — and it's one of the more quietly destructive habits in modern software teams.

The Dashboard That Performs Productivity

Vanity metrics are seductive because they're easy to collect, easy to display, and almost always trending in a direction that feels positive. Commit frequency goes up when developers are busy — which is most of the time. Code coverage climbs whenever someone writes tests — even if those tests are checking things that don't matter. Deployment counts increase whenever your CI/CD pipeline fires — regardless of whether any of those deployments moved the product forward.

The problem isn't that these numbers are useless in isolation. Deployment frequency, for instance, is a real signal in the DORA research — but only when it's paired with change failure rate and mean time to recovery. Stripped of that context and put on a dashboard alone, it just tells you that things are being deployed. It says nothing about whether those things should have been deployed, or whether users noticed, or whether anything broke quietly afterward.

Commit frequency is worse. It's almost purely noise as a standalone metric. A developer who deletes 400 lines of dead code in a single commit did more for the codebase than someone who pushed 40 small commits refactoring variable names. The chart doesn't know the difference.

What Gets Measured Gets Optimized — Into the Ground

Here's where it gets genuinely damaging: teams don't just observe vanity metrics, they start optimizing for them. And once optimization starts, behavior follows.

If your team tracks code coverage as a KPI, developers will write tests that hit lines rather than tests that catch bugs. You'll end up with a codebase that's 90% covered and still surprises you in production every other week. The tests are real. The coverage is real. The safety is theater.

If sprint velocity is a performance indicator, developers will scope tasks smaller to inflate the point count. Estimation becomes a game. The velocity number climbs. The actual output doesn't.

If deployment count signals team health, you'll see more deployments that are smaller, more fragmented, and harder to reason about — not because that's the right architecture, but because it makes the chart look better.

This is Goodhart's Law playing out in real time: when a measure becomes a target, it ceases to be a good measure. Every time.

The Signals That Actually Tell You Something

So what should teams be watching instead? A few alternatives that tend to track closer to reality:

Time-to-value for the user. How long does it take from when a user first encounters a problem to when your product solves it? This cuts across your entire stack — onboarding, performance, UX, reliability. It's harder to game because it's anchored to actual user behavior, not internal activity.

Time-to-resolution on reported issues. Not just open/close time on tickets, but the actual elapsed time from a user experiencing a problem to that problem being fixed in production. This one surfaces your debugging overhead, your deploy pipeline friction, and your team's ability to prioritize accurately — all at once.

Retention by cohort. Are the users who signed up three months ago still here? Are the ones who signed up last month sticking around longer than the previous cohort? Cohort retention is brutally honest. It doesn't care about your feature count or your test coverage. It just tells you whether people found enough value to come back.

Ship velocity on user-facing changes. Not deployments — actual changes that users can see or feel. This is a meaningful distinction. Internal refactors, dependency updates, and config changes are real work, but they're not ship velocity in the sense that matters to your product. Tracking how often you're delivering something a user would notice keeps the team anchored to outcomes.

The Dashboard Redesign Nobody Wants to Have

Stripping vanity metrics off a team dashboard is genuinely uncomfortable. Those numbers going up felt good. Replacing them with slower-moving, harder-to-interpret signals that don't always trend positively — that's a rougher morning standup.

But that discomfort is the point. A metric that makes you feel good regardless of what's happening in your product isn't a metric. It's a mood board.

The teams that ship things users actually care about tend to be the ones who got honest about what they were measuring. They swapped the commit graphs for retention charts. They stopped celebrating deployment counts and started asking what each deployment actually changed for the person on the other end.

It's less satisfying to look at. It's also significantly more useful.

Green charts and leaving users can coexist pretty comfortably. The only way to know which one you have is to measure the right thing.