Two-point sensor calibration calculator
A two-point calibration fits a straight line. Any curvature in the sensor shows up as error in the middle, which the third point exposes.
Converted value
50.013 561
- Gain
- 0.027 122 322
- Offset
- -5.532 953 6
- Error at the check point
- 0.013 561 2
- Error as a share of span
- 0.0136 %
- Engineering units per raw count
- 0.027 122 3
- Raw count for the check value
- 2047.5
Show the working
- Raw reading at the low point = 204
- Actual value there = 0
- Raw reading at the high point = 3891
- Actual value there = 100
- Raw reading to convert = 2048
- Known value at that point = 50
- raw_span = 3891 - 204 = 3687
- actual_span = 100 - 0 = 100
- Gain = 3687 != 0 ? 100 / 3687 : 0 = 0.027 122 3
- Offset = 0 - 0.027 122 3 * 204 = -5.532 95
- Converted value = 0.027 122 3 * 2048 + -5.532 95 = 50.0136
- Error at the check point = 50.0136 - 50 = 0.013 561 2
- Error as a share of span = 100 != 0 ? 0.013 561 2/100*100 : 0 = 0.013 561 2
- Engineering units per raw count = abs(0.027 122 3) = 0.027 122 3
- Raw count for the check value = 0.027 122 3 != 0 ? (50 - -5.532 95)/0.027 122 3 : 0 = 2047.5
Values are shown to six significant figures here; the arithmetic itself runs on exact fractions and is only rounded where a number is printed.
Scenarios & local history
Compare a saved scenario
Recent calculations
The formula
- gain = (actual high - actual low) / (raw high - raw low)
- offset = actual low - gain x raw low
A significant error at mid-scale means the sensor is not linear and needs a curve or more calibration points.
Calibrate at the ends of the range you actually use, not the ends of the sensor range.