How the precision works

Every conversion on this site is computed with exact fractions of arbitrary-precision integers. Rounding happens exactly once, when the number is printed.

The problem with floating point

A JavaScript number is a 64-bit float with about 15–17 usable decimal digits. It also cannot represent most decimal fractions exactly — 0.1 + 0.2 famously comes to 0.30000000000000004. A conversion typically involves at least two multiplications and a division, and each one rounds.

For everyday use that is invisible. It stops being invisible when you chain conversions, when you work near the edges of a tolerance, or when you need more digits than a float has to give.

What this site does instead

Almost every unit in use is defined as an exact ratio to an SI unit. So each of the 754 units here stores its factor as a pair of BigInt integers — a numerator and a denominator — and every conversion is:

1. Parse what you typed into an exact fraction. 0.1 becomes 1/10, not 0.1000…4. 1 1/2 becomes 3/2.
2. Multiply by the source unit's fraction to reach the category's base unit.
3. Divide by the target unit's fraction.
4. Round once, to however many significant digits you asked for.

Steps 1–3 lose nothing at all. Ask for 50 significant digits and you get 50 correct ones, because the number was never approximated on the way there.

Worked examples

Where it cannot be exact — and says so

Two kinds of conversion are genuinely not rational, and the site marks these ≈ Rounded rather than claiming otherwise:

• Anything involving π — degrees to radians, oersteds to A/m, the parsec. π is carried to 75 decimal places, so results are correct far past any display, but the badge stays honest.
• Logarithmic units — decibels, nepers, musical cents, photographic stops. These need a real exponential, computed here in fixed point at 60 digits with ln and exp implemented from series, then rounded to 45.
• Measured constants — the electron mass and the atomic mass unit are experimental values (CODATA 2022), not definitions, so they carry their published uncertainty.

The exactness badge only ever means proven exact. Degrees to arcseconds is exactly 3600 because the π cancels, but the site will not claim that, because it does not try to prove cancellation. It would rather understate than mislead.

Where the factors come from

• The SI Brochure (9th edition) and the 2019 redefinition, which made c, h, k, NA and e exact.
• The 1959 international yard and pound agreement — the source of the exact inch, foot, yard, mile and pound.
• NIST Special Publication 811 for US customary and derived units.
• CODATA 2022 for the handful of genuinely measured constants.

Factors are written in the source as their definitions — 0.3048^3 for a cubic foot, au*648000/pi for a parsec — so each one can be checked against the standard it came from rather than against someone else's calculator. Every unit in all 61 categories is round-tripped through its base unit in the test suite, and the exactness flag is asserted against the definition.

Browse all 61 converters →