Resolution

Resolution is applicable to both output and measurement circuits of power supplies and SMUs.

For power supplies and SMUs, output and measurement resolution are usually specified in absolute units, like μV or nA.

Output Resolution

Resolution of a power supply or SMU output channel is the smallest possible change that can be made to the output voltage or current level. This limitation is imposed because of the finite number of steps that are available in the device DAC circuit.

Output resolution can be calculated by dividing the total span of the output range by the number of possible quantized values the DAC allows.

For example, consider a ±6 V output range when using a 16-bit DAC:

Resolution = (+6 V – (–6 V)) / (2¹⁶–1) = 12 V / 65535 = 183 μV

Measurement Resolution

Measurement resolution of a power supply or SMU is the smallest change in the voltage or current measurement that can be detected by hardware. The resolution for a particular channel is based on the resolution of the ADC used to digitize the measured signal. When taking a measurement of the output, the ideal ADC coerces the actual value to the nearest ADC code. The codes are equally spaced across the measurement range of the ADC, each separated from the previous and next code by the magnitude of the power supply or SMU resolution.

For example, consider a ± 200 μA measurement range when using an 18-bit ADC:

Resolution = (+200 μA - (-200 μA)) / (2¹⁸-1) = 400 μA / 262143 = 1.5 nA

Sensitivity

Sensitivity is the smallest unit of a given parameter that can be meaningfully detected with an instrument under specified conditions. This unit is generally equal to the resolution in the smallest range of a power supply or SMU.