Abstract
A model is presented to evaluate the accuracy of diagnostic tests from data from individuals that are repeatedly tested in time. Repeated measurements from three diagnostic tests for foot-and-mouth disease, applied to vaccinated and experimentally infected cattle, were analyzed. At any time the true disease status of the individuals was unknown, i.e., no gold standard was available. The model allows for correlation between repeated test results, in consequence of the underlying structure for the unknown true disease status, but also by the distribution of the test results conditional upon true disease status. The model also allows for dependence between the different diagnostic tests conditional upon true disease status. Prior information about the structure of the prevalence and the specificity of the tests was incorporated in a Bayesian analysis. Posterior inference was carried out with Markov chain Monte Carlo. Simulated data were analyzed to gain insight into the performance of the posterior Bayesian inference. The simulated data are typical for the expensive and, therefore, modestly sized infection experiments that are conducted under controlled conditions.
Article PDF
Similar content being viewed by others
References
Brocchi, E., Bergmann, I. E., Dekker, A., Paton, D. J., Sammin, D. J., Greiner, M., Grazioli, S., De Simone, F., Yadin, H., Haas, B., Bulut, N., Malirat, V., Neitzert, E., Goris, N., Parida, S., Sorensen, K., and De Clerq, K. (2006), “Comparative Evaluation of Six ELISAs for the Detection of Antibodies to the Non-Structural Proteins of Foot-and-Mouth Disease Virus,” Vaccine, 24, 6966–6979.
Cook, R. J., Ng, E. T. M., and Meade, M. O. (2000), “Estimation of Operating Characteristicsfor Dependent Diagnostic Tests Based on Latent Markov Models,” Biometrics, 56, 1109–1117.
Dahlquist, G. Björck, Å. (1974), Numerical Methods, New Jersey: Prentice-Hall.
Dendukuri, N., and Joseph, L. (2001), “Bayesian Approaches to Modelling the Conditional Dependence Between Multiple Diagnostic Tests,” Biometrics, 57, 158–167.
Engel, B., Bouma, A., Stegeman, A., Buist, W., Elbers, A., Kogut, J., Döpfer, D., and de Jong, M. C. M. (2005), “When Can a Veterinarian Be Expected to Detect Classical Swine Fever Among Breeding Sows in a Herd During an Outbreak?” Preventive Veterinary Medicine, 67, 195–212.
Engel, B., Buist, W., Orsel, K., Dekker, A., de Clercq, K., Grazioli, S., and van Roermund, H. (2008), “A Bayesian Evaluation of 6 Diagnostic Tests for Foot-and-Mouth Disease for Vaccinated and Non-Vaccinated Cattle,” Preventive Veterinary Medicine, 86, 124–138.
Engel, B., Buist, W., and Visscher, A. (1995), “Inference for Threshold Models With Variance Components From the Generalized Linear Model Perspective,” Genetics Selection Evolution, 27, 15–32.
Engel, B., Swildens, B., Stegeman, A., Buist, W., and de Jong, M. (2006), “Estimation of Sensitivity and Specificity of Three Conditionally Dependent Diagnostic Tests in the Absence of a Gold Standard,” Journal of Agricultural, Biological and Environmental Statistics, 11, 360–380.
Gardner, I. A., Stryhn, H., Lind, P., and Collins, M. T. (2000), “Conditional Dependence Between Tests Affects the Diagnosis and Surveillance of Animal Diseases,” Preventive Veterinary Medicine, 45, 107–122.
Gelman, A. (2006), “Prior Distributions for Variance Parameters in Hierarchical Models,” Bayesian Analysis, 3, 515–533.
Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (1995), Bayesian Data Analysis, London: Chapman & Hall.
Georgiades, M. P., Johnson, W. O., Gardner, I. A., and Singh, R. (2003), “Correlation-Adjusted Estimation of Sensitivity and Specificity of Two Diagnostic Tests,” Applied Statistics, 52, 63–76.
Hedeker, D. (2003), “A Mixed-Effects Multinomial Logistic Regression Model,” Statistics in Medicine, 22, 1433–1446.
Hui, S. L., and Walter, S. D. (1980), “Estimating the Error Rates of Diagnostic Tests,” Biometrics, 36, 167–171.
Kitching, R. P., Hutber, A. M., and Thrusfield, M. V. (2005), “A Review of Foot and Mouth Disease Virus With Special Consideration for the Clinical and Epidemiological Factors Relevant to Predictive Modelling of the Disease,” Veterinary Journal, 169, 197–209.
Miglioretti, D. L. (2003), “Latent Transition Regression for Mixed Outcomes,” Biometrics, 59, 710–720.
Qu, Y., and Hadgu, A. (1998), “A Model for Evaluating Sensitivity and Specificity for Correlated Diagnostic Tests in Efficacy Studies With an Imperfect Reference Test,” Journal of the American Statistical Association, 93, 920–928.
Qu, Y., Tan, M., and Kutner, M. H. (1996), “Random Effects Models in Latent Class Analysis for Evaluating Accuracy of Diagnostic Tests,” Biometrics, 52, 797–810.
Redner, R. A., and Walker, H. F. (1984), “Mixture Densities, Maximum Likelihood and the EM-Algorithm,” SIAM Review, 26, 195–239.
Schulzer, M., Anderson, D. R., and Drance, S. M. (1991), “Sensitivity and Specificity of a Diagnostic Test Determined by Repeated Observations in the Absence of an External Standard,” Journal of Clinical Epidemiology, 44, 1167–1991.
Spiegelhalter, D., Thomas, A., Best, N., and Lunn, D. (2003) WinBUGS User Manual. Version 1.4, available at http://www.mrc-bsu.cam.ac.uk/bugs .
Wolfram Research, Inc. (2007), Mathematica, Version 6.0, Champaign, IL.
Yang, I., and Becker, M. P. (1997), “Latent Variable Modelling of Diagnostic Accuracy,” Biometrics, 53, 948–958
Zhang, S., and Jin, J. (1996), Computation of Special Functions, New York: Wiley.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Engel, B., Backer, J. & Buist, W. Evaluation of the Accuracy of Diagnostic Tests From Repeated Measurements Without a Gold Standard. JABES 15, 83–100 (2010). https://doi.org/10.1007/s13253-009-0013-y
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13253-009-0013-y