Shindo Ask a language model to summarise a set of health numbers and it will do something subtly dangerous: it will get them almost right.
It will round 6.2 to 6, which is fine. It will describe a form value of −22 as “significantly negative”, which is fine. And then, once in a while, it will say your resting heart rate rose by four beats when it rose by two, or restate a training load figure it half-remembered from earlier in its own reply. Nothing in the sentence looks wrong. It reads exactly as fluently as the correct version.
For a chatbot that is an acceptable error rate. For something a person uses to decide whether to train hard on a morning they feel flat, it is not — and it is not acceptable in a specific way that matters: the error is invisible at the point of use. The reader has no independent copy of the number to check it against. The app is the copy.
So the rule in Shindo is that the model is never in a position to be wrong about a number, because it is never the one producing it.
The rule#
Every metric — readiness and its four drivers, training stress, fitness, fatigue, form, decoupling, nutrition targets, thresholds, every figure in every card — is computed by deterministic code on the iPhone before any text is generated. The model is then handed those finished numbers and asked for sentences.
It may not compute. It may not re-derive. It may not restate a figure in its own words, re-round one, or infer a new one from two it was given. Its entire job is phrasing.
Stated as a rule that is easy to agree with and easy to violate, which is why it is not only a rule. It is enforced at runtime, after the model replies and before anything reaches the screen.
The guardrail#
A small, pure validator sits between the model’s reply and the app. It receives two things: what the model produced, and the engine’s authoritative values for the same facts. Then it does something blunt on purpose — it overwrites every authoritative field with the engine’s value, whatever the model sent, and records which fields differed.
Note what that is not. It is not “reject the reply if a number is wrong”. It is “the number is now the engine’s, and by the way, here is the drift we logged”. The model’s figure never had standing to begin with; the comparison exists to tell us how often the model tries, not to decide whether it gets to win.
The user-visible consequence of a rejection is deliberately nothing. No error, no warning triangle, no “the AI could not verify this”. A guardrail catch is a neutral state that degrades quietly to engine facts. An athlete reading their morning report has no interest in our internal disagreement, and a red banner would only teach them to distrust a number that is, at that exact moment, the more trustworthy of the two.
From a ban to a vocabulary#
The first version of this enforced the rule at the prompt: the model was simply forbidden to write any digit at all. Anything numeric in its output was drift by definition and got stripped.
That works, and it produces stilted text. “Your form is significantly negative and your sleep was somewhat below your recent average” is not a sentence a coach would say. The numbers are what make coaching text concrete, and banning them to guarantee correctness sacrifices the thing the text was for.
So the ban was replaced by a vocabulary. The model writes placeholders — {tsb},
{sleepHours} — and the guardrail substitutes the engine’s value for each one. The
model gets to decide that a number belongs in a sentence and where; the engine
decides what it is. Two rules enforce it:
- a placeholder that does not resolve to a known fact drops the text it is in, and
- a literal digit typed outside a placeholder drops it too.
The second rule is the load-bearing one. Without it the vocabulary is decorative: the model would use placeholders where convenient and type numbers where not, and we would be back to trusting it.
Named things — a sport, an intensity band, a training phase — go through a whitelist instead. They carry no number, so a wrong one is a wording bug rather than a false fact, and those are reported to us rather than dropped on the reader.
The day the morning report stopped arriving#
This is the part worth writing down, because it is the failure that taught us the most, and it was entirely self-inflicted.
We shipped the placeholder mechanism and the morning report went blank.
Not wrong — blank. Every card, every morning.
The cause was almost funny. We built a vocabulary of placeholder names, enforced it strictly, and never told the model what the names were. So it invented plausible ones. Every card cited a name that did not resolve. Every card was therefore dropped. The result was an empty stack of cards, which was thrown back up the call chain as an unusable reply — which was, technically, the system working exactly as specified.
Declaring the vocabulary to the model was the obvious fix, and it was not the important one. The important question was: why did one bad token cost the entire feature?
Two changes came out of that, and both narrow what a mistake costs — never what is enforced:
The digit check stopped being paranoid. It had been treating Z2, 2-й and
5×1000 as invented metrics. None of those are figures about you; they are the names of
things. A real figure — a form value, a duration in hours — is still drift. The check
now knows the difference.
Failure became per-field instead of per-card. A bad sentence costs that sentence. A bad badge costs the badge. Only a lost title costs the whole card. Before, any one of the three cost all of it.
The rule that was actually broken was not the guardrail’s. It was ours: the cost of a violation was wildly out of proportion to the violation. A strictness that turns a single unresolved token into a blank screen is not safer than one that drops a sentence — it is the same strictness with a worse failure mode, and it will get switched off in a hurry by whoever is on call. Enforcement you cannot afford to keep on is not enforcement.
The one exception, and why it does not weaken this#
There is exactly one place where a model-produced number can become a real value: the weekly training plan. The model proposes an entire week, and a separate validator with nineteen rules — every threshold read back from the engine that owns it — decides whether that week may stand.
The guardrail is deliberately not consulted for the numbers of an accepted week. It would restore the engine’s week one field at a time and quietly undo the acceptance. It still guards all of that week’s text, and it still owns the numbers on every other surface in the app.
That looks like a contradiction and is not, because the guarantee moved rather than disappearing. On every other surface the guarantee is “this number came from the engine”. On the plan it is “this number was ratified by the engine, against limits the engine owns, and if it could not be, you are looking at the engine’s own week”. Both are checkable claims about a mechanism. Neither is a claim about the model behaving. That mechanism has its own post.
What this is worth to a reader#
Almost every training app now has an AI feature, and from outside they look identical: a paragraph of plausible coaching text. The question a reader cannot answer from the outside is which side of the line the numbers in that paragraph came from.
It is a fair question to ask of any of them. Two versions that get you most of the way:
- Do the numbers in the AI text always match the numbers on the charts? Not “are they close” — identical. If a summary says your form is −18 and the chart says −22, the text is being generated rather than reported.
- Does the AI text still work when the model is unavailable? An app whose numbers are computed locally can still show you every metric with the network off. One that cannot was having them written.
Neither test is conclusive. Both are more informative than any marketing page, including ours.
Every metric Shindo computes, with its formula and constants, is on the methodology page — including the four readiness drivers and the exact windows behind fitness, fatigue and form. The engine-versus-model split described here is stated there too, in the section on the rule.
Read next