Data Talks: Let Your Data Do the Decision Making

Most business decisions I've watched get made in small companies are made on vibes with a slide deck attached afterward. Someone felt like churn was up, so they pulled a number that confirmed it. Someone was sure a feature was popular, because the person who built it talks about it a lot. None of that is malicious — it's just what happens by default when a decision has to get made and the data to check it isn't sitting anywhere convenient. The fix isn't a mandate to "be more data-driven." It's removing the friction between "I have a hunch" and "I can check the hunch in ninety seconds," and being honest about the cases where the check itself can't be trusted.

Instrument before you decide, not after

The single highest-leverage thing you can do is capture the event before you need it, not backfill it once someone asks a question. I've built systems where a business question came up — how often are leads going cold before first contact, how often does a scrape return a suspiciously thin result — and the honest answer was "we don't know, and we can't know, because nothing recorded it." That's not a data problem, it's an instrumentation problem, and it's retroactively unfixable for anything that already happened.

Practical version of this: every workflow that matters to the business should emit a timestamped, structured event at each state transition — lead created, lead contacted, lead qualified, lead lost — not just a final status column that overwrites itself. A status field that goes from new to contacted to qualified tells you where a lead is. A log of status_changed events with timestamps tells you how long each stage took, where things stall, and whether that's gotten better or worse over the last quarter. The second one is what actually supports a decision. The first one is what you're left guessing from when nobody thought to log the transition.

This is cheap to do at the point of building the system and expensive to retrofit. If you're building anything — a CRM sync, an intake form, a support pipeline — the instrumentation is not a nice-to-have bolted on for reporting later. It's part of the system's actual design, on the same footing as the feature itself.

Lightweight reporting beats gut checks, but only if it's actually lightweight

A dashboard nobody opens is worse than no dashboard — it's a maintenance burden that gives false confidence that "we're tracking that." The dashboards that hold up are the ones that answer one specific recurring question, are fast to load, and don't require someone to remember a query.

On a lead-qualification system I built, the useful artifact wasn't a general-purpose BI tool — it was a single view answering "which leads came in today, and which of those look worth calling first," refreshed automatically, with a score attached so a human didn't have to reconstruct the reasoning from raw fields each time. That's the bar: does this let someone make the actual decision they need to make, without translation. If the answer requires exporting to a spreadsheet and pivoting, it's not a dashboard, it's a data dump with a URL.

The same principle scaled down: a five-line script that prints "reviews this week: 42, average rating: 4.3, down from 4.6 last week" and runs on a cron job beats a beautiful chart nobody checks. Reporting infrastructure should match the size of the decision it supports. Most small-business decisions don't need a warehouse and a BI layer; they need a number, on a schedule, in a place someone will actually see it.

Thresholds and trend detection instead of "check it when you remember"

The natural failure mode of any manual reporting process is that it only gets checked when something already feels wrong — which means by the time a human notices, the problem has been live for a while. The fix is moving the threshold check into the system itself, so it triggers action instead of waiting to be looked at.

Concretely, this means something like:

if rating_7d_avg < rating_30d_avg - 0.3:
    alert("Rating trending down: {rating_7d_avg} vs {rating_30d_avg}")

if leads_contacted_within_1h / leads_total < 0.5:
    alert("Response time SLA slipping: {pct}% contacted within 1h")

Nothing sophisticated — a rolling average compared against a longer baseline, and a hard floor on a ratio that matters. The value isn't in the statistics, it's in converting "someone has to remember to look" into "the system tells you when it's worth looking." That distinction is most of what separates a business that catches problems in week one from one that catches them in month three, from the same underlying data.

The trap here is threshold fatigue — if every alert fires constantly, people tune it out within a week, and you're back to ignoring the system. Set thresholds off real historical variance, not a round number that felt right, and expect to spend time in the first month tuning false positives out.

Where "trust the data" breaks down

This is the part that gets skipped in most "be data-driven" advice, and it's the part that actually matters if you're the one making the call.

Goodhart's law is not theoretical. The moment a metric becomes the target, people (and systems) optimize the metric instead of the thing it was supposed to represent. Track "tickets closed" and you'll get tickets closed fast and reopened later. Track "leads contacted within an hour" without also tracking outcome quality, and you'll get fast, low-effort outreach that hits the SLA and converts worse. Every metric you build a decision or an incentive around should be paired with a second metric that would catch the obvious way to game the first one.

Survivorship bias shows up in almost every retrospective dataset. If you're analyzing "what did our best customers have in common," you're only looking at the customers who stuck around long enough to become good data points — the ones who churned in month one because of the exact same behavior are invisible in that analysis. Any dataset built from "who's still here" instead of "everyone who started" is systematically biased toward whatever doesn't cause early failure, and it will confidently point you toward the wrong lesson if you don't correct for it.

Small samples lie with total confidence. A rating dashboard showing a jump from 4.2 to 4.8 stars means something different at 3 reviews than at 300. Automated trend detection built on thin data will produce a stream of statistically meaningless alerts that look exactly as urgent as the real ones — this is where knowing your denominator matters as much as knowing your metric.

Data-driven and data-informed are not the same posture. Data-driven treats the number as the decision. Data-informed treats the number as one input a person weighs against context the data doesn't capture — a competitor's outage that inflated your numbers this week, a known seasonal effect, a metric definition that changed mid-quarter and makes the trend line an artifact rather than a signal. I lean toward the second, especially for anything with a real cost to being wrong. Automate the parts where the data is genuinely unambiguous — SLA breaches, hard threshold crossings — and route anything with more nuance to a human with the numbers in front of them, not a human guessing without them.

None of this is an argument against building the instrumentation, the dashboards, or the alerting. It's an argument for building them with the same rigor you'd want applied to the code itself — knowing what a metric can and can't tell you is part of the job, not an asterisk you add after the fact when a decision goes wrong.