Testing Tests of Symmetry to Evaluate Paired Age Agreement

Tests of symmetry evaluate agreement between paired ages with a null hypothesis that disagreeing paired ages are unbiased with respect to agreeing paired ages. There are three formulations, each with specific sensitivities. McNemar’s test is most sensitive to small differences on one side of the diagonal, if there are many cells where these differences accumulate. Evans and Hoenig’s test is most sensitive to an imbalance projecting along at least one of the off-center diagonals. Bowker’s test is most sensitive to even a single paired difference, once the difference is large enough. Using R software, I simulated paired age data with four types of bias, at three levels of precision. McNemar’s test was redundant with Evans and Hoenig’s test, except when it performed poorly in simulations with under- and overaging at different age classes. Bowker’s test was the least likely to falsely reject the null hypothesis (Type I error) but also the least likely to detect bias (Type II error) when even modest levels of variability were simulated. Evans and Hoenig’s test had the best performance overall, and is recommended as the primary test; McNemar's and Bowker’s tests are probably best reserved for datasets that demand their specific sensibilities.