Abstract
Until we have better reference tests for PCD, diagnostic accuracy cannot be meaningfully evaluated or claimed http://ow.ly/4n6VrI
To the Editor:
The primary ciliary dyskinesia (PCD) group in Southampton, UK, led by J.S. Lucas is to be applauded for its recent efforts to improve the accuracy of PCD diagnosis. In two recent articles published by the group in the European Respiratory Journal [1, 2], a large number of PCD referral cases were analysed and various conclusions were drawn regarding diagnostic accuracy, as well as derivation of a clinical prediction rule that was suggested to aid in further referral and investigations for PCD.
The major problem with these data, however, is that the authors have not essentially evaluated diagnostic accuracy. In diagnostic accuracy studies, the ability of one or more tests to correctly identify patients with the target condition is typically expressed as the sensitivity and specificity of the test(s). These are estimated by comparing the test results to the results of the reference standard, the best available method for correctly identifying patients with the target condition.
Diagnostic accuracy studies can be at risk of bias and several sources of bias have been described [3, 4]. Studies at risk of bias because of design deficiencies typically overestimate the indicators of test performance but also limit the applicability of the study findings.
Unfortunately, these recent PCD studies are not at low risk of bias. The authors did not notify the readers of two design deficiencies. The first is that the diagnostic criteria in the study by Jackson et al. [2] (the reference criteria) include the index tests measured. This was also done in the prediction rule study [1]. The resulting bias is known as incorporation bias, and is known to lead to inflated estimates of accuracy and to generous conclusions about test performance.
Furthermore, the calculated estimates of specificity and sensitivity were based on the number of participants who were diagnosed as PCD (or not) by the reference criteria. However, as no gold standard exists in PCD, exclusion of the disease (true negative) is almost impossible to achieve in that situation. This is exemplified in genetic tests, where a negative finding of a PCD-causing mutation does not necessarily confirm that the patient has no PCD, as only 60–70% of the mutations are known.
We feel that the best (and probably only) diagnostic performance indicator that can be meaningfully discussed in these studies is the positive predictive value of the various tests. To associate these tests with the conventional accuracy statistics, sensitivity and specificity, we still need to wait, as indeed suggested (for other reasons) by Haarman and Schmidts [5] in their editorial accompanying one of the papers. Until we have some reference tests that are better than the current bronze or silver standards, diagnostic accuracy cannot be meaningfully evaluated or claimed.
Footnotes
Conflict of interest: None declared.
- Received April 1, 2016.
- Accepted April 11, 2016.
- Copyright ©ERS 2016