Mixed Model ANOVA
Previous Top
Replicate x Sample vs Error Plots / Replicate x Sample F-statistics
A Replicate*Sample vs Error plot depicts a bar for each attribute indicating whether there is
inconsistency in sample differences from replicate to replicate. A colored bar (yellow, orange,
red) means that there seems to be such and a grey bar that there is not.
The color of the bar indicates the corresponding P-value:
Yellow: P-value < 0.05
Orange: P-value < 0.01
Red: P-value < 0.001
The bar size equals the F-statistic for Replicate*Sample interaction
F=MS(Rep*Sample)/MS(Error)
from a 3-way ANOVA with all main effects and 2-way interactions:
Attribute = Rep + Sample + Assessor + Rep*Sample + Assessor*Sample + Rep*Assessor
Note: This analysis and plot is only relevant IF the replications were carried out in different
sessions, one session for each replication or a grouped collection of sessions for each
replication. If the replications were merely carried out in a random order then the Replication
effects (main and interactions) in this analysis are expected to be NS (apart from “chance
significances”).
Note: Specific attributes, assessors and samples may be excluded from the analysis, by
disabling them in the checkboxes.
Sample x Assessor vs Error Plots / Sample x Assessor F-statistics
An Sample*Assessor vs Error plot depicts a bar for each attribute indicating whether there is
inconsistency in sample differences from assessor to assessor. A colored bar (yellow,
orange, red) means that there seems to be such and a grey bar that there is not.
The color of the bar indicates the corresponding P-value:
Yellow: P-value < 0.05
Orange: P-value < 0.01
Red: P-value < 0.001
The bar size equals the F-statistic for Sample*Assessor interaction
F=MS(Sample*Assessor)/MS(Error)
from a 3-way ANOVA with all main effects and 2-way interactions:
Attribute = Rep + Sample + Assessor + Rep*Sample + Sample*Assessor + Rep*Assessor
Note: Formally this analysis is adequate IF the replications were carried out in different
sessions, one session for each replication or a grouped collection of sessions for each
replication.
If the replications were merely carried out in a random order then the Replication effects
(main and interactions) in this analysis are expected to be NS (apart from “chance
significances”) and it will not generally influence the F-test for the Sample*Assessor
interaction in any important way.
Note: Specific attributes, assessors and samples may be excluded from the analysis, by
disabling them in the checkboxes.
Sample vs Sample x Assessor Error Plots / Sample Fs Assessor Error
A Sample vs Sample*Assessor Error Plot depicts a bar for each attribute indicating whether
there is sample differences using ONLY the Sample*Assessor interaction as the error term. A
colored bar (yellow, orange, red) means that there seems to be such and a grey bar that
there is not.
The color of the bar indicates the corresponding P-value:
Yellow: P-value < 0.05
Orange: P-value < 0.01
Red: P-value < 0.001
The bar size equals the F-statistic:
F= MS(Sample)/MS(Sample*Assessor)
from a 2-way ANOVA:
Attribute = Sample + Assessor + Sample*Assessor
Note: This is the analysis of sample differences often used. However, this analysis is
adequate ONLY IF the replications were carried out in a random order OR if the
Replication*Sample interaction effect is negligible. If not, the results reported in the “Sample
in 3-way Mixed Model Plot” should be used. If in doubt, also use the mixed model analysis,
since it will correspond to the usual analysis whenever the assumptions for this are OK!
Note: Specific attributes, assessors and samples may be excluded from the analysis, by
disabling them in the checkboxes.
Sample in 3-way Mixed Model Plots / Sample Fs Replicate and Assessor Error
A Sample in 3-way Mixed Model Plot depicts a bar for each attribute indicating whether there
is sample differences considering the random contributions from as well assessors as
replications.
A colored bar (yellow, orange, red) means that there seems to be such and a grey bar that
there is not.
The color of the bar indicates the corresponding P-value:
Yellow: P-value < 0.05
Orange: P-value < 0.01
Red: P-value < 0.001
The bar size equals the F-statistic from the 3-way mixed ANOVA model:
Attribute = Rep + Sample + Assessor + Rep*Sample + Sample*Assessor + Rep*Assessor
where everything BUT the Sample main effect is considered random. This means that the F-
statistic is given by
F= MS(Sample)/ MS(Denominator)
Where
MS(Denominator)=MS(Sample*Assessor)+MS(Rep*Sample)-MS(Error)
And the denominator degrees of freedom are found by the Satterthwaithe method.
Note: If the Rep*Sample variation is “too small”, that is, MS(Rep*Sample)<MS(Error) then this
effect is assumed to be negligible and the “usual” F= MS(Sample)/MS(Sample*Assessor) is
used instead. So for these attributes, the bars equal the bars in the “Sample vs
Sample*Assessor Error Plot”.
Note: Formally this analysis is adequate IF the replications were carried out in different
sessions, one session for each replication or a grouped collection of sessions for each
replication.
If the replications were merely carried out in a random order then the Replication effects
(main and interactions) in this analysis are expected to be NS (apart from “chance
significances”) and it will not generally influence the F-test for the Sample main effect in any
important way. Hence, this analysis can be seen as the safer approach, IF in doubt about the
situation: It will take the replication variation into account when necessary and in cases where
it is not, it will provide roughly the same answers as the "usual" analysis.
Note: Specific attributes, assessors and samples may be excluded from the analysis, by
disabling them in the checkboxes.
Sample x Assessor 95% LSD values Plots
A Sample x Assessor 95% LSD values Plot simply depicts the sample average values
including an LSD bar for each attribute. The LSD bar should be used as a dynamic
measuring stick for each attribute: Two samples differing more than this bar are significantly
different. The placement of samples on each side of the bar within each attribute is only to
enhance readability, it says nothing about the samples. The LSD bar is based on the analysis
using ONLY the Sample*Assessor interaction as the error term corresponding to the “Sample
vs Sample*Assessor Error Plot”.
Note that NO corrections for multiple comparisons are used here!
Note: This analysis is adequate ONLY IF the replications were carried out in a random order
OR if the Replication*Sample interaction effect is negligible. If not, the results reported in the
“3-way Mixed Model 95% LSD values Plot” should be used. If in doubt, also use the mixed
model analysis, since it will correspond to the usual analysis whenever the assumptions for
this are OK!
Note: Specific attributes, assessors and samples may be excluded from the analysis, by
disabling them in the checkboxes.
Sample x Assessor 95% Bonferroni LSD values Plots
A Sample x Assessor 95% Bonferroni LSD values Plot simply depicts the sample average
values including an LSD bar for each attribute. The LSD bar should be used as a dynamic
measuring stick for each attribute: Two samples differing more than this bar are significantly
different. The placement of samples on each side of the bar within each attribute is only to
enhance readability, it says nothing about the samples. The LSD bar is based on the analysis
using ONLY the Sample*Assessor interaction as the error term corresponding to the “Sample
vs Sample*Assessor Error Plot”. The Bonferroni correction for multiple comparisons is used.
For non pre-planned sample comparisons this is a more correct approach compared to the
un-corrected approach. The corrected approach will always provide a wider LSD bar.
Note: This analysis is adequate ONLY IF the replications were carried out in a random order
OR if the Replication*Sample interaction effect is negligible. If not, the results reported in the
“3-way Mixed Model 95% LSD values Plot” should be used. If in doubt, also use the mixed
model analysis, since it will correspond to the usual analysis whenever the assumptions for
this are OK!
Note: Specific attributes, assessors and samples may be excluded from the analysis, by
disabling them in the checkboxes.
3-way Mixed Model 95% LSD values Plots
A 3-way Mixed Model 95% LSD values Plot simply depicts the sample average values
including an LSD bar for each attribute. The LSD bar should be used as a dynamic
measuring stick for each attribute: Two samples differing more than this bar are significantly
different. The placement of samples on each side of the bar within each attribute is only to
enhance readability, it says nothing about the samples. The LSD bar is based on the analysis
using the 3-way mixed ANOVA model corresponding to the “Sample in 3-way Mixed Model
Plot”. Note that NO correction for multiple comparisons are used here!
Note: Formally this analysis is adequate IF the replications were carried out in different
sessions, one session for each replication or a grouped collection of sessions for each
replication.
If the replications were merely carried out in a random order then the Replication effects
(main and interactions) in this analysis are expected to be NS (apart from “chance
significances”) and it will not generally influence the F-test for the Sample main effect in any
important way. Hence, this analysis can be seen as the safer approach, IF in doubt about the
situation: It will take the replication variation into account when necessary and in cases where
it is not, it will provide roughly the same answers as the “usual” analysis.
Note: Specific attributes, assessors and samples may be excluded from the analysis, by
disabling them in the checkboxes.
3-way Mixed Model 95% Bonferroni LSD values Plots
A 3-way Mixed Model 95% Bonferroni LSD values Plot simply depicts the sample average
values including an LSD bar for each attribute. The LSD bar should be used as a dynamic
measuring stick for each attribute: Two samples differing more than this bar are significantly
different. The placement of samples on each side of the bar within each attribute is only to
enhance readability, it says nothing about the samples. The LSD bar is based on the analysis
using the 3-way mixed ANOVA model corresponding to the “Sample in 3-way Mixed Model
Plot”. The Bonferroni correction for multiple comparisons is used. For non pre-planned
sample comparisons this is a more correct approach compared to the un-corrected approach.
The corrected approach will always provide a wider LSD bar.
Note: Formally this analysis is adequate IF the replications were carried out in different
sessions, one session for each replication or a grouped collection of sessions for each
replication.
If the replications were merely carried out in a random order then the Replication effects
(main and interactions) in this analysis are expected to be NS (apart from “chance
significances”) and it will not generally influence the F-test for the Sample main effect in any
important way. Hence, this analysis can be seen as the safer approach, IF in doubt about the
situation: It will take the replication variation into account when necessary and in cases where
it is not, it will provide roughly the same answers as the “usual” analysis.
Note: Specific attributes, assessors and samples may be excluded from the analysis, by
disabling them in the checkboxes.
Further reading:
P. B. Brockhoff, Statistical testing of individual differences in sensory profiling, Food Quality
and Preference 14 (2003) 425–43.