Walk into almost any modern operations center and the first thing you notice is not the engineers. It is the dashboards. Large displays show CPU utilization, request latency, queue depth, storage consumption, error rates, deployment frequency, cost trends, security alerts, and dozens of business KPIs. Every metric updates continuously. Every graph suggests control. Yet when something goes wrong, experienced engineers often ask a surprisingly simple question: “Which of these numbers actually explains the problem?”
This is the paradox of modern data platforms. We have become extraordinarily good at collecting operational data, but far less disciplined about identifying which signals should influence engineering decisions.
The consequence is subtle. Our dashboards frequently know more than the people responsible for operating the system. And that is precisely the problem.
A dashboard should reduce uncertainty. Instead, many increase it.
Consider a real-world scenario familiar to many platform engineers. A payment processing platform experiences a sudden increase in authorization latency. The system runs on Google Cloud. Transactions arrive through Pub/Sub, are processed by Cloud Run services, written into BigQuery for analytics, and monitored through Cloud Monitoring and Cloud Logging.
Within minutes, the operational dashboard begins to fill with information. CPU utilization remains healthy. Memory usage is stable. Network throughput fluctuates slightly. BigQuery ingestion continues successfully. Cloud Run instances are scaling normally. The Pub/Sub backlog increases. Application latency rises. Customer complaints begin to appear. Nothing looks obviously broken. Everything looks slightly different. This is where conventional thinking often fails.
Most teams respond by expanding observability. They add more charts, create new metrics, instrument additional services, and enrich dashboards with increasingly detailed telemetry. The underlying assumption is straightforward: more visibility should improve diagnosis. But visibility is not the same as understanding.
Every additional metric introduces another interpretation. Every new dashboard competes for attention. Eventually, engineers spend more time navigating telemetry than reasoning about system behavior. The dashboard becomes a data lake disguised as a user interface.
Research in decision science has repeatedly demonstrated that increasing information beyond a certain point often reduces decision quality by increasing cognitive complexity rather than reducing uncertainty [1]. Observability systems are not immune to this phenomenon.
The engineering challenge is therefore different from what many organizations assume. The objective is not maximizing observability data. The objective is minimizing decision uncertainty. Those are not equivalent goals.
An alternative perspective begins by treating dashboards as decision-support systems rather than reporting tools. Instead of asking, “Which metrics can we display?” Ask a different question. “What operational decision should this dashboard enable within the next sixty seconds?”
The difference appears small, but it fundamentally changes dashboard design. Imagine redesigning the payment platform around that question. The on-call engineer rarely needs thirty graphs during an incident.
The first decision is much simpler.
Is the platform healthy enough to continue processing payments?
Answering that question may require only a handful of carefully selected operational signals: end-to-end transaction latency, successful authorization rate, queue growth, downstream dependency availability, and data freshness.
Everything else becomes supporting evidence. Notice that this approach does not eliminate telemetry. It organizes telemetry around decisions instead of components. This distinction aligns naturally with the perspective we have been exploring at Data S2.
In real-time systems, every additional piece of context carries a cost. It consumes computational resources, increases operational complexity, and, perhaps most importantly, competes for the engineer’s attention.
The challenge is identifying the smallest set of reliable signals capable of supporting the operational decision. Not because other metrics lack value. Because not every metric deserves equal importance during an incident.
A lightweight Google Cloud implementation illustrates this principle well. Cloud Run services emit structured application events into Cloud Logging while infrastructure metrics flow into Cloud Monitoring. Pub/Sub exposes queue depth, and BigQuery records analytical history for offline investigation. Rather than presenting these sources independently, a decision-oriented dashboard derives a small number of operational indicators representing system health.
Detailed logs, traces, and infrastructure metrics remain available. They simply appear when engineers need explanation, not before they have identified the operational question.
This architecture separates diagnosis into two phases. The first reduces uncertainty. The second explains causality. Many dashboards attempt both simultaneously. The result is often neither. There are practical trade-offs.
Highly summarized dashboards risk hiding subtle anomalies that detailed telemetry might reveal. Conversely, dashboards exposing every available metric often overwhelm engineers during high-pressure situations. The appropriate balance depends on operational objectives, incident response maturity, and latency requirements.
The important point is intentionality. Every metric displayed should justify its existence by improving a specific engineering decision. Several implementation mistakes repeatedly undermine this objective.
One is treating dashboard design as a visualization exercise rather than an operational workflow. Beautiful charts rarely improve incident response if they fail to guide decisions.
Another mistake is organizing dashboards around services instead of customer outcomes. Infrastructure rarely fails according to organizational boundaries.
Perhaps the most common error is assuming that because telemetry exists, it belongs on the dashboard. Modern platforms generate virtually unlimited operational data. Engineering discipline lies in deciding what not to display.
Industry practice increasingly supports this direction. Google’s Site Reliability Engineering principles emphasize Service Level Indicators because they capture user-visible system behavior rather than infrastructure detail [2]. Observability Engineering similarly argues that observability exists to answer unknown questions during production failures rather than to maximize telemetry collection [3].
These ideas point toward the same conclusion. Good dashboards are not comprehensive. They are selective. Their purpose is not to show everything the platform knows. Their purpose is to help engineers know enough to act confidently. Perhaps that is how we should evaluate dashboards in the future. Not by asking how many metrics they display. But by asking how quickly they enable the next correct engineering decision. Because if your dashboard consistently knows more than the people using it, perhaps the problem is not the engineers. Perhaps the dashboard has forgotten who it was built for.
References
[1] Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic Decision Making. Annual Review of Psychology.
[2] Beyer, B., Jones, C., Petoff, J., & Murphy, N. Site Reliability Engineering: How Google Runs Production Systems. O’Reilly Media, 2016.
[3] Majors, C., & Fong-Jones, L. Observability Engineering. O’Reilly Media, 2022.