Sensitivity Equations
Below are descriptions of each of the utility-free sensitivity measures that Netica calculates. First are some notes for interpreting the descriptions.
Definition: In the definitions, "belief" means posterior probability (i.e. conditioned on all findings currently entered). In the names of the various measures "real" refers to the = 4 && typeof(BSPSPopupOnMouseOver) == 'function') BSPSPopupOnMouseOver(event);" class="BSSCPopup" onclick="BSSCPopup('X_PU_expected_value.htm');return false;">expected value of continuous nodes, or = 4 && typeof(BSPSPopupOnMouseOver) == 'function') BSPSPopupOnMouseOver(event);" class="BSSCPopup" onclick="BSSCPopup('X_PU_discrete.htm');return false;">discrete nodes which have a real numeric value associated with each state "expected value" means to take the expectation over a quantity.
Range: The minimum and maximum values that this measure can take on.
Compare: A quantity which is useful to compare the value of this measure against (perhaps to express this measure as a percentage).
Equation: Note that all the conditionals should include all findings already entered into the network, so P(q) is really P(q|E), P(q|f) is really P(q|f,E), etc.
Notation:
Q is the query variable
F is the varying variable
q is a state of the query variable
f is a state of the varying variable
Xq is the numeric real value corresponding to state q
SUM~q means the sum over all states q of Q. It applies to
the whole expression following.
MIN~q, MAX~q are similar to SUM~q
E(Q) is the expected real value of Q before any new findings
E(Q|f) is the expected real value of Q after new finding f for node F
V(Q) is the variance of the real value of Q before any new findings
H(Q) is the entropy of Q before any new findings
RMS is "root mean square", which is the square root
of the average of the values squared.
Minimum Belief
Definition: Minimum belief that each state q of Q can take due to a finding at F. This provides a value for each state.
Range: [0, P(q)] P(q) if Q is independent of F
Compare: P(q)
Equation: Pmin(q) = MIN~f P(q|f)
Maximum Belief
Definition: Maximum belief that each state q of Q can take due to a finding at F. This provides a value for each state.
Range: [P(q), 1] P(q) if Q is independent of F
Compare: P(q)
Equation: Pmax(q) = MAX~f P(q|f)
RMS Change of Belief
Definition: The square root of the expected change squared of the belief of state q of Q, due to a finding at F This provides a value for each state. This is the standard deviation of P(q|f) about P(q) due to a finding at F, with the finding at F distributed by P(f).
Reference: Spiegelhalter89 & Neapolitan90,p394. They call the square of this quantity simply "variance".
Range: [0, 1] 0 if Q is independent of F
Compare: P(q)
Equation: sp(q) = sqrt (Vp(q))
Vp(q) = SUM~f P(f) [P(q|f) - P(q)] ^ 2
"Variance" of Node Belief (named "Quadratic Score" in older versions of Netica)
Definition: The expected change squared of the beliefs of Q, taken over all of its states, due to a finding at F.
Reference: Spiegelhalter89 & Neapolitan90,p394. They call this "variance" (for them it comes out the same as Vp(q) because they just use 2-state nodes).
Range: [0, 1] 0 if Q is independent of F
Equation: s2 = SUM~f SUM~q P(q,f) [P(q|f) - P(q)] ^ 2
Minimum Real
Definition: The lowest that the expected real value of Q could go to,
due to a finding at F.
Requires: Node Q is continuous, or has real number state values defined.
Range: (-infinity, E(Q)] E(Q) if Q is independent of F
Compare: E(Q) = SUM~q P(q) Xq
Equation: mmin = MIN~f E(Q|f)
Maximum Real
Definition: The highest that the expected real value of Q could go to, due to a finding at F.
Requires: Node Q is continuous, or has real number state values defined.
Range: [E(Q), infinity) E(Q) if Q is independent of F
Compare: E(Q)
Equation: mmax = MAX~f E(Q|f)
RMS Change of Real
Definition: The square root of the expected change squared in the expected real value of Q, due to a finding at F. This turns out to be the same as the square root of the variance reduction of expected value.
Requires: Node Q is continuous, or has real number state values defined.
Range: [0, V(Q)] 0 if Q is independent of F
Compare: E(Q) and maybe V(Q)
Equation: sm = sqrt (Vm)
Vm = SUM~f P(f) [E(Q|f) - E(Q)] ^ 2 = Vr
Variance Reduction of Real
Definition: The expected reduction in variance of the expected real value of Q due to a finding at F. This turns out to be the square of RMS Change of Real.
Requires: Node Q is continuous, or has real number state values defined.
Range: [0, V(Q)] 0 if Q is independent of F
Reference: Pearl88,p323. What he says is C(T|X) is actually C(T|X)-C(T).
Var mapping: T->Q, X->F, C->V, t->q and Xq
Compare: V(Q)
Equation: Vr = V(Q) - V(Q|F) = Vm
V(Q) = SUM~q P(q) [Xq - E(Q)] ^ 2
V(Q|f) = SUM~q P(q|f) [Xq - E(Q|f)] ^ 2
E(Q) = SUM~q P(q) Xq
Entropy Reduction (Mutual Information)
Definition: The mutual information between Q and F (measured in bits). The expected reduction in entropy of Q (measured in bits) due to a finding at F.
Range: [0, H(Q)] 0 if Q is independent of F
Reference: Pearl88,p321. He has sign of I(T,X) backwards.
Var mapping: T->Q, X->F, I(T,X)->I
Compare: H(Q)
Equation: I = H(Q) - H(Q|F)
= SUM~q SUM~f P(q,f) log (P(q,f) / [P(q) P(f)])
Note that the log is base 2, which is traditional for entropy and mutual information, so that the units of the results will be "bits".