Two velocity readings and a stopwatch. Subtract, divide, and our own kinematics calculator reports an acceleration of 0.0750 m/s².
Every digit after the zero is invented. Not wrong, exactly — the arithmetic is correct. Simply not determined by anything the reader typed in.
What a written number claims
Writing a measurement as 2.50 claims the true value lies between 2.495 and 2.505. This is the whole point of significant figures.
The usual rule for carrying that through a calculation is to keep as many significant digits in the answer as the least precise input had. Our science pages implement it: they read the string as typed, count its digits, and print the result at that width.
So we audited it. Rather than apply the rule, we treated every input as the interval it claims to be, ran the engine at every corner of that box, and asked how many digits of the answer stay put. A digit counts as justified when the uncertainty is under half a unit in its place, which is the textbook criterion.
Anything beyond that is a digit the calculator invented.
Multiplication and division: the rule holds
Across 768 multiplications and divisions through our physics kernel — Ohm's law, kinetic energy, density, momentum, wave speed, heat transfer — only 34 showed more digits than the inputs justify. That is 4.4 per cent, and never by more than 2 digits.
For the operations most calculators spend their time on, the convention works. Good thing, too, because the rest of this is less reassuring.
Subtraction: the rule fails
Subtracting two numbers that are close together destroys precision, and it does so invisibly. The leading digits cancel; the uncertainties do not.
Across 120 cases where two independently supplied quantities are subtracted, 73 overstated the precision. That is 60.8 per cent, by up to 3 digits.
The rule cannot see this coming, because it only looks at the inputs. Two four-figure measurements are two four-figure measurements whether their difference is large or vanishing.
The worked case
Return to the acceleration. Velocity 10.00 m/s falling to 9.85, over 2.000 seconds.
Each of those is a good measurement, and the rule says the answer keeps three significant figures. But 10.00 means anything from 9.995 to 10.005, and 9.85 means anything from 9.845 to 9.855, so the difference is anywhere from 0.140 to 0.160 — a spread of more than a tenth of itself.
Carry that through and the acceleration lies between 0.070 and 0.080. The number of justified digits is 0. Not one figure of the answer is determined, and the page prints three.
An honest report would not round the number, because there is no digit to round to. It would state that the acceleration is between 0.070 and 0.080, or that the measurement was too coarse to answer the question.
What this means for our own tools
Of the 25 calculators running on this kernel, 20 propagate input precision, meaning if you type fewer digits you get fewer back. The other 5 use a fixed width, and for at least one of those it is the right call — a Punnett square returns exact ratios, not measurements, and rounding them would be worse.
The subtraction problem is different, and it is ours. Two of our pages will happily show three or four figures on a difference that justifies none: the kinematics calculator solving for acceleration, and the thermodynamics one computing free energy as a difference. The rule they implement is the standard rule, and the standard rule is what fails. Naming it here is the first step to fixing it.
The defect this audit actually found
Setting out, we expected to find our tools claiming precision they had not earned. We found the opposite as well, and it was worse.
The kernel's formatting function worked out how many decimal places to print from the number of digits before the decimal point, treating anything below 1 as having one. That describes the leading zero rather than the first significant digit, which in 0.075 is two places further in.
So values between 0.01 and 1 came out short, and some came out empty. Across 144 sampled combinations of value and requested width, 13 rendered a non-zero quantity as 0 or 0.0. Asked for the acceleration above to one significant figure, the function returned a bare zero.
That range is not a corner case for these tools. Concentrations, accelerations, probabilities and absorbances all live between 0.01 and 1.
How it survived is more interesting than the bug itself. Our seven tests on this function straddled the gap perfectly: small values, 0.005 and below, took a scientific-notation path, and every other case was 1 or larger. Nothing tested the gap between. All seven still pass, unchanged, alongside six new cases covering it — and the fix has shipped.
Something we expected and did not find
We went in expecting failures to cluster on answers with a small leading digit. "Three significant figures" spans a factor of ten in real precision — 1.00 is uncertain by half a per cent, 9.99 by a twentieth of that — and the rule cannot tell them apart.
The data does not show it. Answers beginning with 1 overstated 9.9 per cent of the time, and answers beginning with 8 overstated 12.8 per cent, with several digits never failing at all. The reasoning still looks sound and the measurement does not support it, so it stays out of the conclusions and in the sources.
What to do with a number on a screen
Ask what the least precise input was, and stop reading there. A calculator has no idea how carefully you measured, and it will not volunteer that its last four digits are decoration.
Be suspicious whenever a calculation subtracts two similar quantities — a change in temperature or velocity, a difference in energy, a mass found by difference. This is the classic setup, common in the school and lab work these tools are for.
When it happens, the fix is measurement rather than arithmetic. No amount of care with digits recovers precision that the subtraction removed — you need readings further apart, or an instrument with more resolution.
The significant-figures rule is mostly fine. Across 768 multiplications and divisions through our own kernel it overstated precision only 4.4 per cent of the time. Subtraction is the exception: when two close numbers are subtracted their leading digits cancel but their uncertainties do not, a fact the rule cannot see because it only inspects the inputs. In our tests that happened 60.8 per cent of the time, and in the worked case an acceleration displays three significant figures when the inputs justify zero — the true value sits anywhere between 0.070 and 0.080. So find the least precise input and stop reading there, and treat any difference of two similar quantities as suspect. The fix is not better rounding; it is a better measurement.
Checking your own numbers
The script is committed with this guide and re-runs the whole audit. Our kinematics calculator and thermodynamics calculator are the two that carry the subtraction described above, the Ohm's law calculator is a straightforward product where the rule holds, and the rounding calculator handles the mechanics of getting a number to a given number of figures. For the wider problem of precision claimed against precision earned, see our guides on confidence intervals and standard deviation against standard error. For the same trap in a different subject, see what a Punnett square cannot tell you.
- Every figure is computed by a script committed alongside this guide, using
science-kernel.js— the module our own physics, chemistry and biology calculators run on. Nothing is quoted from an external table, so this guide cannot drift from the tools it audits. - ⚠️ THIS IS A SELF-AUDIT. The calculators being measured are ours, and the measurement uses the same module they run on. Treat it as we would want a reader to treat any organisation marking its own homework.
- ⚠️ THE FORMATTING DEFECT DESCRIBED ABOVE WAS FIXED BEFORE THIS GUIDE PUBLISHED, in the same repository, with six regression cases that were themselves falsified by mutation. The audit figures do not depend on it: they compare input precision against the interval the inputs define, and never call the display function. Re-running the measurement across the fix leaves every number identical.
- ⚠️ THIS MEASURES DISPLAY PRECISION, NOT CORRECTNESS. Every value the kernel computes is arithmetically right. The only question asked here is how many of its digits are worth printing.
- ⚠️ THE CONVENTION IS ALLOWED ONE DIGIT OF SLACK. The significant-figures rule is a rule of thumb and is known to be approximate at the boundary, so a result is counted as overstated only when the gap EXCEEDS one digit. Counting the ordinary off-by-one as a defect would inflate every number here and bury the real finding.
- ⚠️ THE FIRST METRIC WAS WRONG AND WAS THROWN AWAY. It counted digits that agree after rounding, which confuses "known" with "happens to round the same way": 1.00 × 1.00 scored one justified digit purely because 0.99 and 1.01 both round to 1. It fired on 32 per cent of ordinary multiplications. The replacement is the textbook criterion — a digit is justified when the uncertainty is under half a unit in its place — and the multiplication figure fell to 4.4 per cent.
- ⚠️ A HYPOTHESIS WAS TESTED AND REFUTED, and is recorded rather than quietly dropped: failures were expected to cluster on answers with a small leading digit, and they do not. Answers beginning 1 overstated 9.9 per cent against 12.8 per cent for those beginning 8, with unbalanced bins. The guide makes no claim from it.
- ⚠️ INPUTS ARE READ AS TYPED, INCLUDING TRAILING ZEROS, because that is what the pages do. An earlier draft of the script passed numbers instead of strings, which silently dropped trailing zeros and understated the input precision — the same mistake in the opposite direction.
- ⚠️ THE INTERVAL IS THE CONVENTIONAL READING OF A WRITTEN NUMBER, half a unit either side of the last significant digit. It is not an instrument uncertainty budget: a real instrument may be worse than this, never better.
This audits how many digits our own calculators display against how many their inputs justify. It is not a statement about any other tool, not a metrology standard, and not a substitute for the uncertainty analysis a laboratory or examination board requires.
View author profile → · Editorial policy
