The closeness of agreement between independent measurements obtained under stipulated conditions. Precision quantifies the spread or dispersion of repeated measurement results.
Precision reflects how consistently an instrument produces the same result when measuring the same quantity under the same conditions. High precision means the measurements cluster tightly together, regardless of whether they are close to the true value. Precision is often expressed numerically as standard deviation or variance of a set of repeated measurements.
In metrology, precision encompasses two related but distinct concepts: repeatability and reproducibility. Repeatability refers to precision under identical conditions (same operator, same instrument, same environment, short time frame), while reproducibility refers to precision under changed conditions (different operators, instruments, or locations). Both are important for understanding the overall capability of a measurement system.
For calibration management, precision matters because it determines the minimum uncertainty that can be achieved with a given instrument. Even after an instrument is adjusted for perfect accuracy, its precision places a floor on the quality of measurements it can deliver. When evaluating whether an instrument is suitable for a particular measurement task, both its accuracy and precision must be considered relative to the tolerance requirements of the application.
In aerospace calibration labs, precision is critical when calibrating torque wrenches used for critical fasteners on aircraft engines. A lab may perform 10 consecutive measurements of a 50 N⋅m reference standard, obtaining readings between 49.98 and 50.03 N⋅m. The standard deviation of 0.015 N⋅m demonstrates excellent precision, essential for AS9100 compliance where measurement repeatability directly impacts flight safety. In medical device manufacturing, precision becomes vital during blood pressure monitor calibrations. When testing a sphygmomanometer against a 120 mmHg reference, repeated measurements showing values scattered between 119.2 and 120.8 mmHg indicate poor precision that could lead to misdiagnosis. Common precision failures include: inadequate instrument warm-up time causing drift between measurements, environmental variations during multi-point calibrations, and technician technique inconsistencies. During ISO 17025 audits, assessors frequently identify precision issues through control chart analysis, particularly when labs cannot demonstrate measurement repeatability within stated uncertainties. Poor precision often manifests as failed proficiency testing results or customer complaints about inconsistent calibration certificates, leading to corrective actions and potential accreditation suspension.
ISO/IEC 17025:2017 addresses precision in sections 7.2.1.1 and 7.7.1, requiring laboratories to demonstrate measurement repeatability through ongoing validity checks and control of measurement processes. The standard mandates that precision be considered when estimating measurement uncertainty per Annex A. AS9100D references precision through measurement system analysis requirements in section 7.1.5.2, demanding statistical validation of measurement repeatability. ISO 13485:2016 requires precision validation in section 7.6, particularly for measuring equipment used in medical device manufacturing where measurement variability directly impacts patient safety. GUM (ISO/IEC Guide 98-3) extensively covers precision in Type A uncertainty evaluation (section 4.2), requiring statistical analysis of repeated observations. ANSI/NCSL Z540.3-2006 addresses precision in measurement assurance programs (section 10), requiring documented evidence of measurement repeatability. During audits, assessors examine control charts, repeatability studies, and uncertainty budgets to verify that laboratories properly characterize and monitor precision. Common audit findings include inadequate precision studies, missing control charts, and failure to incorporate precision components into uncertainty calculations.
CalibrationOS captures precision data through its Statistical Process Control (SPC) module, automatically calculating standard deviations and control limits from repeated calibration measurements. The Uncertainty Budget feature incorporates precision components as Type A uncertainties, computing repeatability contributions from historical calibration data. During multi-point calibrations, the system tracks measurement scatter at each test point, flagging excessive variation that may indicate instrument instability. The Control Chart module generates real-time precision monitoring through X-bar and R charts, alerting technicians when measurement repeatability exceeds established limits. Audit Trail functionality maintains complete precision data history, enabling assessors to verify measurement consistency over time. Certificate generation automatically includes precision statements and uncertainty contributions derived from repeatability studies. The software's Measurement Assurance Program feature schedules periodic precision checks using reference standards, tracking long-term repeatability trends. Dashboard analytics provide precision performance metrics across instrument types and technicians, supporting continuous improvement initiatives and regulatory compliance demonstrations during ISO 17025 assessments.
Precision is the degree to which repeated measurements under unchanged conditions show the same results. It is quantified using statistical measures like standard deviation and indicates the consistency of a measurement system.
Precision is measured by taking multiple readings of the same measurand under controlled conditions, then calculating the standard deviation or range of those readings. Lower spread indicates higher precision.
This article is licensed CC BY-SA 4.0. Share, adapt, and reuse with attribution to calibrationos.com/glossary/precision.
Industry benchmarks, best practices, and calibration tips — delivered to your inbox.
Free calibration management software with audit-ready tracking, uncertainty budgets, and compliance tools.
Start Free — No Credit Card