---
title: "Sometimes the numbers won't match, and that's okay: fixing measurement paralysis in app growth"
description: "Why more data isn’t always the answer"
language: "en"
publishedAt: "2026-11-11T14:00:00.000Z"
updatedAt: "2026-11-11T14:00:00.000Z"
authors:
  - name: "Hannah Parvaz"
    url: "https://www.revenuecat.com/blog/author/hannah-parvaz"
category: "Growth"
categoryUrl: "https://www.revenuecat.com/blog/growth"
canonical: "https://www.revenuecat.com/blog/growth/measurement-paralysis"
---

# Sometimes the numbers won't match, and that's okay: fixing measurement paralysis in app growth

Why more data isn’t always the answer

## Table of contents

- [ATT broke the illusion of perfect attribution](#att-broke-the-illusion-of-perfect-attribution)
- [The numbers are going to disagree](#the-numbers-are-going-to-disagree)
  - [Identify your normal disagreement](#identify-your-normal-disagreement)
- [Build the measurement system around decisions](#build-the-measurement-system-around-decisions)
  - [Match the evidence to the decision](#match-the-evidence-to-the-decision)
  - [How Opal kept it simple](#how-opal-kept-it-simple)
- [Use blended metrics as your business reality](#use-blended-metrics-as-your-business-reality)
  - [Beware: blended truth still hides averages](#beware-blended-truth-still-hides-averages)
- [What good measurement actually looks like](#what-good-measurement-actually-looks-like)

Subscription apps have more data than ever, but more data does not automatically produce better decisions. The real skill is knowing what each number can tell you, where it is likely to be wrong, and when the evidence is good enough to act.

There’s a meeting I’ve sat in (some version of) too many times.

Paid acquisition says customer acquisition cost (CAC) is $42 when finance says it’s $67. Meta says one campaign is outperforming everything else, when the mobile measurement partner (MMP) is much less enthusiastic. Product is celebrating an 18% improvement in trial starts, but the revenue data suggests trial-to-paid is falling…

The meeting inevitably ends with someone opening a dashboard to clarify the numbers, only to find it sparks three new debates.

Everyone has data, and everyone can defend their number. Yet, **no one can figure out what to do next. **And this isn’t usually happening because a company has no data. It frequently happens when a company has *lots of data*, but no shared understanding of it.

This measurement dysfunction is one of the most underestimated barriers to app growth.

## **ATT broke the illusion of perfect attribution**

If you were running mobile growth in 2021 and you’re still here, you’re already battle-tested. Apple’s App Tracking Transparency (ATT) framework became a reality with iOS 14.5 in April 2021, pushing the industry into a much more aggregated and privacy-preserving measurement world.

Lucas Moscon’s [guide to SKAdNetwork (SKAN) ](https://www.revenuecat.com/blog/growth/skadnetwork-guide-subscription-apps)describes the pre-ATT period as a time when user-level [metrics](https://www.revenuecat.com/blog/growth/activation-metrics) made deterministic attribution much more accessible. Then came ATT, SKAN postbacks, conversion values, anonymity thresholds, shorter windows, and far less visibility at a user and creative level.

There was a fog around data: everyone was trying to work out what numbers they could still see (not many), what they could send back to the platforms (not much), and which parts of the old playbook still worked (again, not many).

But the industry adapted. We learned SKAN. We added more tracking. We got better at first-party data. Platforms built modeled attribution (hello, Meta’s Aggregated Event Measurement, or AEM). Teams got big into [web funnels](https://www.revenuecat.com/blog/growth/web-to-app-funnels/), incrementality tests, cohort analysis, marketing mix modeling (MMM) and more sophisticated internal business intelligence.

Five years later, the problem is almost the inverse of where we started: most subscription **businesses now have** **more measurement** than they did before ATT, but **less alignment on what the numbers actually mean**.

## **The numbers are going to disagree**

Today, each team working on a subscription app has gotten better at measuring their own part of the journey, but the connections between those views have definitely not improved at the same speed.

UA teams look at platform models and MMPs; product has richer behavioral analytics; finance keep their own revenue view; and subscription teams get better cohort and renewal data. But this segregated measurement** **is how you end up with five sophisticated systems and a basic argument about whether CAC is $42 or $67.

That does not mean the post-ATT measurement stack failed. It means we have to **stop expecting each layer to produce the same answer.**

![](https://cdn.sanity.io/images/c3qnx9b0/production/9ce8c49c657e173de7996807c5ccf157aad43835-1280x720.png)

*One customer journey can create several valid views because each system is answering a different question.*

One example is Meta: for one of our recent clients, Meta is reporting less than a third of the conversions we see in App Store Connect across the same period. That’s a huge gap if you look at the two numbers and **assume they are supposed to match**.

We don’t panic about it. We don’t believe the campaigns suddenly became 3.5x worse. Most importantly, we don’t spend the week trying to make Meta and App Store Connect display the same conversion total. What we *do *do is understand the gap:

- Is it broadly stable?
- Does it widen or narrow when we change how spend is deployed?
- Does Meta still correctly identify the campaigns and creatives that are relatively stronger or weaker?
- What happens to the business-level numbers when we scale?
Once you know those answers, an under-reporting platform can still be very useful. It just **cannot be treated as a complete representation of reality.**

This is something [David Vargas](https://www.linkedin.com/in/davidvargasmontiel/) has talked about - his LinkedIn is basically a running catalog of what iOS measurement looks like in the real world. [In one example](https://www.linkedin.com/posts/davidvargasmontiel_last-week-we-had-a-good-case-with-aem-in-share-7254400761793576960-O66u/), RevenueCat showed 90 free trials while Meta AEM reported 33. In [another](https://www.linkedin.com/posts/davidvargasmontiel_skan-can-work-better-than-aem-for-some-activity-7313468014610280448-Kb2N), the App Store produced a materially lower cost per trial than AEM.

The point is not that *Meta* is uniquely bad. It’s that **discrepancy in your data is normal.**

### **Identify your normal disagreement**

The way we solve this at Aperture is by defining what ‘normal disagreement’ looks like. E.g. if Meta usually reports 35% fewer trials than your baseline, that’s your normal disagreement. Seeing 100 trials on your baseline tool and 65 on Meta isn’t concerning. But if that gap suddenly becomes 70% disagreement, then that *is *worth investigating. **A stable mismatch is very different from a new mismatch.**

This is the part a lot of measurement frameworks miss. Reliable data doesn’t mean identical numbers. It requires the team to understand why the numbers differ, roughly by how much, and what would count as an anomaly.

In short, the useful question is not “which dashboard is right?” It is: **what decision am I trying to make with this number?**

## **Build the measurement system around decisions**

A lot of companies experiencing this kind of measurement pain respond by building another dashboard to bring all the numbers into one place. And sometimes — *on rare occasions *— that is useful.

But if the numbers are already inconsistent, then a new dashboard is only a means to display those inconsistent numbers together.

I would start somewhere less exciting: **write down the decisions your team repeatedly needs to make, like:**

- Should we increase channel spend next week?
- Which ad set gets the next $1,000?
- Did the paywall test improve the business?
- Is the US cohort genuinely better than the UK cohort?
- Can we afford to acquire a customer at $60?
- Is this channel adding customers or mostly taking credit for demand we already had?
Then, for each decision, decide four things:

1. **The fastest useful signal:** what can move quickly enough to help you act?
1. **The downstream guardrail:** what stops you from declaring a false win?
1. **The business anchor:** what blended outcome should ultimately move if the strategy is working?
1. **The evidence threshold:** how certain do you need to be given the size and reversibility of the decision?
Here’s what that looks like for four common decisions:

| Decision | Fast signal | Downstream guardrail | Business anchor |
| --- | --- | --- | --- |
| Allocate creative spend | Platform CPT / CPA | Trial-to-paid, early value | Blended CAC / revenue |
| Roll out onboarding change | Activation / trial start | Paid conversion, refunds, retention | Cohort revenue |
| Change price or paywall | Purchase conversion | Plan mix, renewal, refund rate | Realized revenue / contribution |
| Reallocate major channel budget | Attributed CPA + cohort quality | Incrementality / longer-term value | Blended CAC, ROAS, total revenue |

The same measurement stack should use different signals for different kinds of decisions. Only after sorting that would I worry about whether the company needs another reporting tool.

### **Match the evidence to the decision**

Once you accept that no single metric can do every job, the next question becomes much easier: **how much evidence does this decision actually require?**

I’ve had versions of this debate for years. In a [Singular Growth Masterminds conversation](https://www.singular.net/blog/first-million-users/), John Koetsier and I talked about teams wanting to judge every product experiment on lifetime value. The intention is good, and LTV is important, but you cannot necessarily use lifetime value to judge a week-long experiment. You would never learn anything quickly enough.

We must also consider the stakes of each decision. Looking at our four common decisions above — each differs drastically. A $500 **creative-allocation** decision tomorrow does not require the same evidence as moving $3 million of annual spend between channels next week. For the first decision, platform data can be perfectly adequate, if you understand the usual reporting gap and you are watching the blended business outcome.

For a **paywall or pricing change**, I want a [controlled experiment](https://www.revenuecat.com/feature/experiments) and downstream subscription guardrails. For a **large channel reallocation**, I want cohort quality, blended performance and, where possible, incrementality evidence. For **financial planning**, I care far more about realized financial data and explicit assumptions.

![](https://cdn.sanity.io/images/c3qnx9b0/production/9ecac87133c5c1823126d22a2dd155086a0148e2-1280x920.png)

*The bigger and less reversible the decision, the stronger the evidence should be.*

This is also why directional data is not dirty data. The obsession with perfect measurement can become another form of paralysis.

If Meta consistently finds one creative at half the reported cost per trial of everything else, I do not need to wait a year to put more budget behind it. But I *do* want to know whether the creative is bringing in a cohort that converts and retains sensibly, and whether blended economics remain healthy as spend increases.

The standard of evidence rises with the cost of being wrong.

### **How Opal kept it simple**

A good example is Opal. On [Sub Club, Opal’s CEO, Kenneth Schlenker](https://www.revenuecat.com/blog/growth/kenneth-schlenker-opal-sub-club-podcast), shared that the seven-person team had reached $5 million annual recurring revenue (ARR) ([and they’ve since doubled that](https://www.speedinvest.com/knowledge/scaling-smart-how-opal-built-a-10m-arr-business-in-just-2-years)).

Opal invested in paid acquisition early, but rather than trying to build an enormous measurement framework, they focused on Day 8 return on ad spend (ROAS) as a practical growth signal. Kenneth described the approach as **“just focusing on this one metric: Day 8 return on ad spend.”**

That number wasn’t the final answer to every question about the business. But it was useful because it was available quickly enough to make decisions, and close enough to the economics they cared about.

> **Tip:**
> There’s an important **difference between an imperfect metric and a useless metric**. Growth teams get into trouble when they forget that.

## **Use blended metrics as your business reality**

When the attribution gets noisy, I like to retrace my steps back to the boring numbers.

How much did you spend? How many customers did you acquire? How much revenue did those customers generate?

That blended view is your anchor.

If the business spent $200,000 during a period, and acquired 4,000 new paying customers, the [blended acquisition](https://www.revenuecat.com/blog/growth/measure-hybrid-monetization) cost is $50. There will still undoubtedly be debate about which channels caused those customers, which captured demand, or how organic growth contributed. But before you have that debate, you at least know the economic boundary the models have to fit inside.

- Platform data helps us make decisions inside the platform
- Attribution helps us understand where growth appears to be coming from
- Blended performance tells us **whether the business as a whole is actually becoming more efficient**

![](https://cdn.sanity.io/images/c3qnx9b0/production/765b8e91bb9e3f44927e209834db241d03aa7a97-1200x664.png)

*Attribution models can disagree with one another. The blended business totals give those models an economic boundary.*

Blended reporting does not solve attribution. It doesn’t tell you whether Meta caused customer #4,382 to subscribe, but it does tell you what you put into the machine and what came out.

Perhaps even more importantly, it also gives you somewhere to sanity-check the stories your channels are telling you. E.g. if every ad platform claims more conversions than the business acquired in total, you know the channel reports cannot all be interpreted literally.

I often think of this as working from the outside in. Start with the size and economics of the whole pie, then use attribution and segmentation to work out how it’s being divided. If you start with the individual slices, you risk scaling a channel that looks brilliant in isolation, while the overall business barely moves.

### **Beware: blended truth still hides averages**

There is a catch to everything I have just said about blended metrics: **averages can hide important differences too.**

Blended CAC is a fantastic anchor. But it is not a segmentation strategy.

[Eric Crowley made this point years ago](https://www.revenuecat.com/blog/growth/best-metrics-for-understanding-subscription-app-growth) when discussing LTV: “The problem with this metric, and lots of other metrics, is that they are derived from an average.”

You’ve probably heard about ‘tourists’ and ‘locals’ — tourists may subscribe quickly, but also churn quickly. On the other hand, locals can become deeply engaged, long-lived customers. Put both groups into one average and you can end up with a number that represents neither.

The same problem appears everywhere in subscription growth. A blended trial-to-paid rate can hide one country converting brilliantly and another badly; overall LTV can hide radically different monthly and annual subscribers; and average CAC can hide an audience that is cheap to trial but expensive to pay.

Time creates another complication. RevenueCat Charts distinguishes [realized LTV](https://www.revenuecat.com/blog/growth/what-is-lifetime-value-ltv-apps) from predicted LTV: one reflects value already generated; the other estimates future value. A Day 30 realized cohort is observed but immature. A 12-month predicted LTV is more complete, but it’s an estimate rather than observed data. Both are useful: finance may care about contribution after fees and refunds, while growth may need an earlier gross-revenue signal to decide whether to scale tomorrow.

Across all metrics, it’s never as simple as “this is always the best one to use”. The hierarchy is not ‘blended good, segmented bad’. It’s more:

Use blended metrics to understand **whether the business economics work**. Then **segment to understand *****why***** they work**, where they are breaking, and which levers you should pull next.

## **What good measurement actually looks like**

After all of this, you can see that good measurement is actually pretty unglamorous. And it’s never as clear as it seems.

A trial-conversion lift can look great until those users cancel more often. A channel can look efficient until you zoom out and realize the blended economics are moving in the wrong direction. None of that makes the original number useless. It just means no single metric gets to answer every question.

The goal is not to make every system agree. The goal is to know:

- What each number is useful for
- When to sense-check the number against the blended picture
- Which numbers help you operate day to day
- Which ones tell you whether the business is healthy
- Which decisions are important enough to require extra evidence
The measurement stack will keep changing. We made it through ATT, SKAN, disappearing user-level signals, and several years of being told attribution was dead. There will be another version of this problem.

But if everyone can look at the data and know what to do next, the measurement is doing its job.

---

## Related posts

- [The blended ARPU framework: how to measure hybrid monetization](https://www.revenuecat.com/blog/growth/measure-hybrid-monetization)
- [Your app's growth has stalled: what now?](https://www.revenuecat.com/blog/growth/app-growth-stalled)
- [Why most activation metrics don’t predict who will stay (and what to use instead)](https://www.revenuecat.com/blog/growth/activation-metrics)