Here’s an uncomfortable question for any health app team: if your users hit their health goals, does your DAU go up or down?
For a lot of health products, the honest answer is down. The user who finally sleeps well stops checking their sleep score at 2 a.m. The one whose recovery routine is dialed in doesn’t need to stare at the dashboard. Meanwhile the user opening your app eleven times a day might not be your healthiest, they might be your most anxious, a pattern well documented enough to have a name in sleep tracking: orthosomnia. If success can look like fewer opens and failure can look like more, a metric that counts opens is not measuring what you think it’s measuring.
This is the trap of borrowed metrics. DAU, session length, and opens-per-day came from social and content products, where attention literally is the business. Health apps adopted them by default. But much of a health product’s value is delivered while the app is closed: data collected passively, scores computed in the background, a nudge that lands and gets acted on without a session ever starting. Measure that product with attention metrics and you’ll optimize it into something it shouldn’t be. Here’s a framework for measuring it honestly instead.
First, classify your app by where value is delivered
Session-value products can keep their session metrics; ambient-value products can’t, and most health apps are more ambient than their dashboards admit.
- Session-value: meditation, guided workouts, therapy exercises, food logging. The value happens inside a session, so sessions completed is a real value metric and DAU is a fair proxy. If this is you, the standard playbook mostly works.
- Ambient-value: recovery and readiness, sleep insight, passive monitoring, risk detection. The value is that the product is watching and interpreting continuously. Opens are optional; the data collection itself is passive. For these products, attention metrics measure the interface, not the value.
- Hybrid: most real apps. A fitness app has workout sessions (session-value) and a recovery score (ambient-value). Measure each half with its own yardstick, not one blended DAU.
The rest of this piece is about the ambient half, because that’s where the borrowed metrics lie the loudest.
The activation gate nobody puts on the dashboard
For a health app, activation isn’t signup or first open. It’s the moment data starts flowing, and there are two gates in front of it.
Gate one: the permission grant. Before anything else, the user must grant health permissions, and grants are granular and partial. A user who installs but grants nothing is not activated in any meaningful sense; your product is running on air. Permission grant rate per data type is the first number that belongs on the dashboard, and it’s also the one most shaped by your onboarding design.
Gate two: the data connection. Permissions granted still doesn’t mean data exists. The user needs a source that actually produces it: a phone that’s carried, a wearable that’s worn and synced. Connection rate, and what kind of source connected, defines what your product can even do for that user.
Only after both gates comes the moment that deserves the name activation: time to first personalized insight. Not first open, but the first time your product tells the user something true about them specifically. The faster that moment arrives, the more of the value loop a new user experiences before their initial motivation fades, which is why cold-start design is a retention lever, not a nice-to-have.
A useful audit: take your current funnel dashboard and check whether “granted permissions” and “data source connected” appear anywhere between install and your activation event. If they don’t, your funnel has a hole exactly where health apps lose the most users.
Data continuity is the real habit metric
The retention question for an ambient product isn’t “did they open the app today,” it’s “is their data still flowing,” because that’s the actual subscription.
Streaks and open-based habit metrics measure interface behavior. The relationship an ambient health product actually has with its user is quieter: the user keeps granting access, keeps wearing the device, keeps letting the data flow, because what comes back is worth it. Call the measurable version data continuity: the share of days with usable data, independent of opens.
Two sharp edges in the definition, both of which bite in production:
- Continuity is per stream, not per user. A user whose sleep data flows nightly but whose workout data died with an abandoned chest strap is two different retention stories, and one blended number averages them into fiction. Compute continuity per data type and alert on the streams your product actually depends on.
- “Stopped arriving” is not “stopped existing.” Health data reaches your pipeline in bursts, not in real time: the wearable syncs to its companion app on its own schedule, and on iOS your app reads HealthKit only when the system wakes it or the user opens something. A user who ignores their phone over a long weekend can look churned on Monday and fully continuous on Tuesday, when three buffered days land at once. Key continuity to when data was recorded, not when it arrived, and finalize each day only after a short lag window. Otherwise your silent-churn trigger fires on exactly the relaxed, healthy users this table tells you to leave alone.
Read retention through that lens and the states sort out differently than DAU suggests:
| User state | Opens | Data flowing | What it actually means |
|---|---|---|---|
| Healthy ambient user | Low | Yes | Product is working; don’t “re-engage” them into anxiety |
| Anxious checker | Very high | Yes | High DAU, at-risk user; success is calming usage |
| Silent churn | Low | Stopped | Effectively gone, whatever the install base says |
| Session user | High | Yes | Fine, if yours is a session-value product |
Two consequences. First, silent churn is your earliest warning: data stops flowing days or weeks before the uninstall, when a device gets abandoned in a drawer or a permission gets revoked. It’s the leading indicator, and it’s invisible on a DAU chart. Second, re-engagement campaigns triggered on “hasn’t opened the app” will actively target your healthiest users. The right trigger is “data stopped,” which is a different list of people. We dug into why users abandon health apps in the churn piece; continuity is how you see it coming in time to act.
Measure acted-on insights, not delivered ones
For engagement quality, count the insights that changed behavior, not the notifications that got sent or even opened.
An ambient product’s engagement isn’t sessions, it’s the loop: the product notices something, tells the user, and the user does something differently. So instrument the loop, not the megaphone. A bedtime nudge “engaged” a user only if bedtime actually moved. A recovery warning mattered only if the user trained lighter. This is measurable precisely because you have the data stream: the behavior after the nudge is in the same data that triggered it.
That standard also disciplines your notification strategy. If a nudge never changes the downstream signal, it isn’t engagement, it’s noise you’re training users to ignore, and gamification mechanics that reward opens rather than behavior fail the same test. Fewer, better-timed, measurably-acted-on nudges beat a louder stream on every metric that matters, including the one most teams forget to check: whether users who receive more notifications actually retain better, or just tolerate you longer before the uninstall.
The attribution trap
A naive acted-on rate flatters itself, because the conditions that trigger a nudge are the same conditions that resolve on their own. Three effects inflate it:
- Regression to the mean. You nudge when a metric is extreme: bedtime drifted late, recovery tanked. Extreme values tend to drift back toward normal with or without your nudge, so “behavior improved after we nudged” is partly the metric doing what extreme metrics do.
- Selection effects. The users most likely to act on a nudge are the motivated ones who were improving anyway. Comparing actors to non-actors mostly measures pre-existing motivation.
- Novelty. Almost any new nudge moves behavior for two weeks. The honest question is what it does in week eight.
None of this makes the metric worthless; it makes the naive version overconfident. The fixes are standard and cheap at product scale: hold out a random slice of eligible users from each nudge and compare their drift against the nudged group, define “acted on” against the user’s own baseline for similar days rather than against yesterday, and read acted-on rates as a relative signal for comparing nudges and timing, not as an absolute claim of impact. A team that reports “our bedtime nudge beats holdout by n points at week eight” knows something. A team that reports “68% of nudges were acted on” mostly knows how regression to the mean feels.
The north star: does the number you promised actually move?
The only metric that ultimately defends a health product is outcome movement, and everything above exists to serve it.
Every health app makes a promise: better sleep, better recovery, less stress, a healthier weight. The user’s data contains the verdict. Cohort your users by whether the promised metric is actually improving, and you get the most honest read of product-market fit a health product can have, because retention follows outcomes in this category. People stay with what works on their body; no streak mechanic outruns a product that doesn’t.
Carry the attribution discipline down to this layer too, because outcome cohorts have a survivorship problem of their own. The users a health product fails are the ones most likely to leave, so “average sleep improved among month-three users” is partly a measurement of who stayed around to be measured. Read outcomes on fixed signup cohorts with the churned users included, against each user’s own baseline, or your north star will keep reporting that the product works right up until growth stalls.
That gives you a measurement stack, top to bottom:
| Layer | Metric | Question it answers |
|---|---|---|
| Activation | Permission grant rate, connection rate, time to first insight | Can the product work for this user, and how fast did they feel it? |
| Habit | Data continuity | Is the real subscription alive? |
| Engagement quality | Acted-on insights | Are we changing behavior or making noise? |
| North star | Outcome movement | Is the promise coming true? |
| Business | Retention, conversion | Following from all of the above |
None of this says attention metrics are worthless; it says they’re downstream. DAU is a fine diagnostic for the interface. It’s just not the product. For a health app, the product is a loop of data in, insight out, behavior changed, outcome moved, and each link in that loop has its own number.
One honest caveat: don’t go looking for industry benchmarks for these. Published benchmarks exist for opens and retention, not for continuity or acted-on rates, and your numbers will vary by category, audience, and data sources. Establish your own baseline, then compete with it.
Instrumenting this, concretely. The whole framework reduces to eight events and two computed metrics. Events: permission_granted and permission_revoked (per data type), source_connected and source_disconnected (per source type), first_insight_delivered (one per user, the activation timestamp), day_with_data (a daily flag per user per data type, keyed to the recorded date and finalized after the lag window), insight_delivered, and insight_acted (with the baseline window and holdout flag attached). Computed: continuity (share of the last 28 days with day_with_data, per stream) and acted-on rate vs holdout per nudge type. If your analytics can answer “which users’ continuity dropped below 50% this week,” you have the dashboard this article is arguing for.
The teams that instrument this loop don’t just measure better; they build differently, because the dashboard finally rewards the same thing the user does: a product that works with the app closed.
The prerequisite for all of it, of course, is that the data layer itself is trustworthy: continuity can’t be measured over a pipeline that drops or double-counts the data it’s supposed to be counting.