How Accurate Are AI Health Coaches With Wearable Data? The Best Model Scored 73%. Most Scored Under 60%

More than 40 million people a day ask ChatGPT about their health [1]. Oura, WHOOP, Google and Apple all ship AI coaches built on the data from their devices, and most health apps are building or buying one. At the same time, the patient safety organisation ECRI ranked misuse of AI chatbots as the number one health technology hazard of 2026 [2].

Both facts are true at once, and the question for anyone building on wearable data is narrower than “is AI good at health?”. It is: when a model reasons over one person’s own data, how often is it right? Four studies published in the past year measured exactly that. Their answer is more useful than a single accuracy number: how a product gives the model the data matters as much as which model it uses.

Which AI model is most accurate with wearable data?

On the largest test built from real users’ data, Meta’s WearableQA, the answer as of September 2026 was Gemini 3.1 Pro, by a wide margin [3]:

Each question had ten options, so guessing scores 10%. Rankings like these change with every model release, and other benchmarks order the same models differently: in Dexcom’s diabetes benchmark, GPT-5 had the best average [4]. The four studies of the past year together give a clearer picture than any one leaderboard:

The pattern across them is consistent. Systems built for a narrow task, with the right data and tools, approach expert level. Even the strongest general models, answering open questions about a person’s data, are wrong between a quarter and a half of the time, and the errors are not random.

Why do AI coaches get wearable data wrong?

Five failure modes recur across the studies.

1. They do arithmetic in their heads. A language model predicts text; it does not calculate unless it is given something to calculate with. In WearableQA, GPT-5.4 answered 51.2% correctly from text data and 71.3% when it could run Python on the same data, a gain of 20 points from one change [3]. In DM-Bench, questions requiring a glucose variability metric called MAGE averaged 8.9% across models [4].

2. They answer from the textbook, not from the user. WearableQA compared pairs of signals related by definition, such as steps and active energy, with pairs whose relationship can only be learned from the person’s own data, such as resting heart rate and stress. Gemini 2.5 Pro scored 51.5 points higher on the definitional pairs, GPT-5.4 48.2 points and GPT-4o 36.0 points [3]. A coach that does this gives the population answer to a personal question, and sounds just as confident either way.

3. They need history, and most products do not send it. Removing a user’s long-term record cut Claude Opus 4.6’s accuracy on questions that depend on history from 75.9% to 39.8%, while questions about a single window were unaffected, 58.1% against 58.0% [3]. “Is this normal for me?” cannot be answered from today’s numbers. We covered how to fit months of data into a model’s context in health data and the LLM context window.

4. Charts make them worse. Showing GPT-5.4 the data as plots instead of text dropped its accuracy from 51.2% to 36.6% [3]. Models read numbers better than pictures of numbers.

5. They are inconsistent, and they under-escalate. In CausalHealthBench, tool-using agents reversed their causal verdicts across runs on the same data, and the weakest systems recommended unsafe self-experimentation and missed dangerous drug interactions and critical lab results [5]. In DM-Bench, models omitted escalation criteria and urgency levels on alert and triage tasks, and safety scores were lowest for people with type 1 diabetes [4].

How do you build an AI health coach that gets it right?

The benchmarks point to the same architecture:

• Compute in code, explain in language. Averages, trends, baselines and correlations should be calculated by software and handed to the model as values or tools. The model’s job is to choose what matters and explain it. Researchers behind WEQA, which routes each question to sensor analysis tools before the model answers, report 24% higher accuracy than model-only and agent baselines [7].
• Send history as structured text. Tables of daily values with dates and units, plus a precomputed personal baseline, beat both raw samples and charts.
• Check every number in the answer. Before a response reaches the user, compare each figure it quotes with the source data. A mismatch is a hallucination you can catch automatically.
• Put safety outside the model. Red-flag thresholds, such as a very high resting heart rate or low blood oxygen, belong in deterministic rules that override whatever the model says.
• Stay inside wellness unless you mean not to. Encouraging habits is general wellness; telling a user they have a condition is not. We covered where the FDA draws that line in the 2026 general wellness guidance. Some data providers also restrict how their data may be used with AI at all, which we compared in can you feed wearable data to an AI.

How do you test an AI health coach before launch?

None of the benchmarks above can tell you how your coach, with your prompts and your data pipeline, performs. A fixed test suite can. These are the cases worth writing, each aimed at a failure the research has already found:

Score each category separately rather than averaging them into one number. Treat any miss on red flags or medications as a release blocker. Rerun the suite on every model upgrade and every prompt change, because a coach that passed last month can fail after a provider updates its model.

AI coaches are not unreliable by nature. They are unreliable when they are asked to be calculators, statisticians and clinicians at once. The evidence from the past year is clear about the fix: let software do the computing, give the model the person’s history, keep safety rules outside it, and test it like the health product it is.