Precision

NI Digital Multimeters

Precision

Precision is a measure of the stability of the DMM and the repeatability of its measurement results for the same input signal. Resolution is a measure of the precision. Precision is given by:

Precision = 1 – |XnAv(Xn)|/|Av(Xn)|

where

Xn = the value of the nth measurement

Av(Xn) = the average value of the set of n measurements

For example, if you are monitoring a constant voltage of 1 V, and your measured value changes by 20 µV between measurements, then your measurement precision is:

Precision (1 – 20 µV/1 V) x 100 = 99.998%

Precision is most valuable when you are using the DMM to calibrate a device or perform relative measurements.