Vanity reports: 5 questions to find out if your agency measures what matters

Every month the agency report arrives: reach climbing, impressions in the thousands, a likes chart with a green arrow. And at the end of the month, the question the report never answers: how many clients did this bring in?

Vanity metrics vs. business metrics

A vanity metric is any number that goes up without your business getting better. Impressions, reach, followers, likes — they measure activity, not results. They’re not useless as intermediate signals; they’re useless as conclusions.

Business metrics answer owner questions: how many leads came in, how many became clients, what each one cost, how much they billed. They’re harder to measure — and that’s exactly why vanity reports exist: they’re the easy way out.

The 5 questions

At your next monthly meeting, ask these questions. The quality of the answers tells you everything.

1. “How many leads did we generate this month, and from which channels?” If the answer mixes “interactions” with leads, there’s a problem. A lead is an identifiable contact with demonstrated interest — a name and a phone number, not a click.

2. “What did each lead cost, per channel?” Total spend divided by leads, channel by channel. If Google costs €12 per lead and Instagram €80, that decides where the next euro goes. Anyone who doesn’t know this number is allocating budget by gut feeling.

3. “How many of those leads became clients?” The question that separates agencies from systems. It requires someone to connect the marketing to your sales records — CRM, calendar, invoicing. If nobody built that connection, nobody knows whether the marketing works. Full stop.

4. “What did the tracking miss?” A technical question with a revealing answer. Between ad blockers and iOS restrictions, browser-only tracking loses a meaningful slice of conversions. Anyone measuring seriously knows their blind spots and compensates for them (server-side tracking); anyone who isn’t says “the numbers are all in Analytics”.

5. “What are we cutting this month?” Real measurement produces decisions — and the hardest decision is cutting. If in twelve months of reports nothing was ever cut, nobody is optimizing; they’re renewing the retainer.

The final test

A useful report reads in five minutes and ends with three decisions. A vanity report reads in thirty seconds of scrolling and ends with “we’ll keep monitoring”.

If the five questions went unanswered, the problem may not even be the agency — it may be the measurement infrastructure nobody ever built. But it’s yours: it’s your budget that’s deciding blind.