Three calculations that go wrong without saying so — and a box for your own.
The two arithmetics. Double precision on this page is IEEE-754 binary64 — the arithmetic of this browser, of spreadsheets, and of most programs by default; it keeps about sixteen significant digits. Exact is rational arithmetic on integers of any size, with no rounding at any step. Everything is computed here, in your browser, as you open it. Two is exactly what this page computes in — those two, and no others. The box further down also prints what single precision would store, which is one rounding of the finished answer rather than a third computation, and it says so on the line. And there are arithmetics this page does not use at all: decimal in pocket calculators and ledgers, float32 and bfloat16 in graphics and machine learning. The appendix shows how to find out which one a tool is actually using.
Below are three short calculations. Each has an answer that is known and agreed. Each, run in ordinary machine arithmetic, returns something else — and nothing in the output says so. The three are fixed examples — open each to see its run, computed in this browser as you open it. The boxed section further down is where you enter a calculation of your own.
The sequence below is defined by a simple rule. In exact arithmetic it approaches 5. Run it in double precision — the arithmetic your browser, your spreadsheet, and most programs use by default — and watch it.
x₀ = 4, x₁ = 4.25 xₙ₊₁ = 108 − (815 − 1500 / xₙ₋₁) / xₙ
It does not crash, oscillate, or flag anything. It converges — smoothly, confidently — to 100. Nothing in the output distinguished this from a correct run. The result was not random: it is the stable point of the arithmetic you did not choose, not of the calculation you wrote.
This sequence is exactly the successive powers of 1/1.618033988…, computed by subtraction. In exact arithmetic it decreases toward zero forever. In double precision:
a₀ = 1, a₁ = 1 / 1.6180339887498949 aₙ₊₁ = aₙ₋₁ − aₙ
For a while it is right to every digit shown. Then the error grows by a fixed factor each step, until the sequence stops shrinking and begins to diverge, flipping sign from one step to the next — an oscillation that grows without bound, the opposite of the true decay. The two step numbers where that happens are read off the run above, not stated here — they depend on this engine's arithmetic, and a page that typed them in would be asserting what it can compute.
The next value is a single arithmetic expression. Its exact value is about −0.8274. Written two algebraically identical ways and evaluated in double precision, it gives two different results, neither near the truth.
Same expression, same inputs, same arithmetic — only the order of operations differs. A number that changes when you rearrange the algebra was never resting on solid ground.
None of this is a defect in any tool. It is what fixed-precision arithmetic does, and it has been documented for decades. The one thing missing is a warning — so supply your own. Given a number you care about, do three things to it and watch whether the answer moves:
| check | what to do | how, concretely | read it as |
|---|---|---|---|
| coarser | recompute at lower precision | Python numpy.float32(x); a spreadsheet's single precision; carry fewer digits | if the answer moves a lot, precision is load-bearing |
| exact | recompute in exact or higher precision | Python fractions.Fraction; mpmath at 50 digits; a CAS | a large move means the fast path lost something that mattered |
| nudge | perturb an input by one unit in the last place | Python math.nextafter(x, math.inf) | if the answer jumps, the computation is unstable there |
Here is the exact check run for you on the first calculation — the same recurrence, once in double precision and once in exact rational arithmetic:
Reading it. Judge "moves" against the size of the quantity, not in absolute terms — and expect a small, harmless move on any irrational answer, since √2 or πr² has no exact fractional form. What you are watching for is a large or uncontrolled move: that means the number is unstable and should not be trusted as printed. A number that holds steady under all three is numerically stable — which is necessary, not sufficient.
One caution. These checks test the arithmetic, not the intent. They cannot tell you the formula was the right one: a wrong formula, computed stably, passes all three and is still wrong. Stability is not correctness.
A value here is a piece of arithmetic: numbers joined by + − * /, brackets, powers, and the few functions listed below. Type one — your own, or one of the examples — and the box runs the three checks from the table above on it: what double precision stores, the same value at single precision, the exact value where it can be known here, and the nudge — the last number in your expression moved by one unit in its last place, and the whole thing recomputed. The two arithmetics are the ones named at the top of the page.
examples: 0.1 + 0.2 · 0.1 + 0.2 − 0.3 · 1 / (0.1 + 0.2 − 0.3) · 1/3 three times · 1/49 · √2 · π · sin π · (√2)²
The exact line appears only when the expression is ordinary arithmetic (computed here as an exact fraction) or is one of a few known constants; for anything else the true value must come from an exact tool such as mpmath. Every figure is computed in your browser; nothing is looked up.
A few one-line probes tell you which arithmetic a tool actually uses, before you rely on any answer it gives.
If a tool returns 0.1 + 0.2 as exactly 0.3, or reports that 2⁵³ + 1 ≠ 2⁵³, or prints an irrational to more than about sixteen correct digits, it is not using ordinary double precision — it is decimal, exact, or symbolic underneath, and these particular failures will not bite it in the same way.
Four more probes this page cannot run for you — they read conventions, not arithmetic, and a convention lives in the tool you use, not in this one. Type each into that tool:
| type | one answer means | the other means |
|---|---|---|
log(2) | 0.6931… — natural logarithm (programming languages, mathematics) | 0.3010… — base 10 (spreadsheets, most calculators; natural is ln there) |
sin(180) | 0 — degrees (most calculators as switched on) | −0.8011… — radians (every programming language) |
2.5 rounded to a whole number | 2 — round half to even (IEEE, Python) | 3 — round half away from zero (spreadsheet ROUND, calculators) |
1/3 | count the digits shown — that is the tool's display limit, which is not its precision; a spreadsheet shows at most fifteen, and stores fifteen when you paste | |
One more, found while this page was being benched against Python: pow is not required by IEEE-754 to be correctly rounded, only recommended, and the golden table's n = 2 row differs by one unit in the last place between a browser's engine and CPython (…d60 against …d61; the correctly rounded value is CPython's). The relative error printed on that row is therefore the engine's, not the arithmetic's — which is the page's subject, met in its own table.
None of these is an error; each is a choice the tool's builders made, and both choices are in daily use. A number carried from one tool to another carries these choices with it — and the same expression, typed into two tools, can return two correct answers to two different questions. If your tool uses a comma as the decimal mark, 0,980 and 0.980 are one number; a file that crosses between the two conventions may not be.