Recommended reading (quick reference): 

 

How to Answer Your Clinical Questions More Efficiently. Weinfeld, JM., Finkelstein, K.

A Simple Method for Evaluating the Clinical Literature. Flaherty, RJ.

 

Checklist to determine what the paper is about:

 

Why was the study done (what clinical question did it address)?

What type of study was done?

  • Primary research (experiment, randomized controlled trial, other controlled clinical trial, cohort study, case-control study, cross-sectional survey, longitudinal survey, case report or case series)?
  • Was the study design appropriate to the broad field of research addressed (therapy, diagnosis, screening, prognosis, causation)?
  • Was the study ethical?

Checklist for the methods section

Was the study original?

Who was the study about?

  • How were the subjects recruited?
  • Who was included or excluded from the study?
  • Were the subjects studied in “real life” circumstances?

Was the design of the study sensible?

  • What intervention or other maneuver was being considered?
  • What outcomes were measured, and how?

Was the study adequately controlled?

  • If a “randomized trial”, was randomization truly random?
  • If a cohort, case-control or non-randomized comparative study, were the controls appropriate?
  • Was the assessment of outcome (or, in a case-control study, allocation of cases) blind?

Was the study large enough, and continued for long enough, and was follow-up complete enough, to make the results creditable?

Checklist for statistical aspects of the paper

 
Have the authors set the scene correctly?
  • Have they determined whether their groups are comparable and, if necessary, adjusted from baseline differences?
  • What sort of data have they got and have they used appropriate statistical tests?
  • If statistical tests in paper are obscure, why have the authors chosen to use them?
  • Have the data been analyzed according to the original study protocol?

Paired data, tails, and outliers

  • Were paired tests performed on paired data?
  • Was the two tailed test performed whenever the effect of an intervention could conceivably been a negative one?
  • Were outliers analyzed with both common sense and appropriate statistical adjustments?

Correlation, regression, and causation
  • Has correlation been distinguished from regression and has the correlation coefficient (‘r value’) been calculated and interpreted correctly?
  • Have assumptions been made about the nature and direction of the causality?

Probability and confidence
  • Have ‘p values’ been calculated and interpreted appropriately?
  • Have confidence intervals been calculated and do the authors’ conclusions reflect them?
Have the authors expressed their results in terms of the likely harm or benefit which an individual patient can expect, such as:
  • Relative risk reduction?
  • Absolute risk reduction?
  • Number needed to treat?
  • Odds ratios?

Checklist for paper validating a diagnostic or screening test

 
  1. Is the test potentially relevant to my practice?
  2. Has the test been compared with a true gold standard?
  3. Did this validation study include an appropriate spectrum of subjects?
  4. Has work up bias been avoided?
  5. Has observer bias been avoided?
  6. Was the test shown to be reproducible both within and between observers?
  7. What are the features of the test as derived from this validation study?
  8. Were confidence intervals given for sensitivity, specificity, and other features of the test?
  9. Has a sensible ‘normal range’ been derived for these results?
  10. Has this test been placed in the context of other potential tests in the diagnostic sequence for the condition?

Assessing the effects of an intervention

 
  Outcomes Event Total
 
  Yes No  
 
Control Group a b a+b
Intervention Group c d c+d
 
 
Risk of outcome event(s) in control group = a/a+b = x
Risk of outcome event(s) in intervention group = c/c+d = y
Relative risk reduction  = x/y = a/(a+b)x(c+d)/c
Absolute risk reduction = x-y
Number needed to treat = 1/x-y
 
Odds ratio (likelihood ratio) =
 
Odds of outcome even vs odds of no event in intervention group
Odds of outcome events vs odds of no outcome event in control group
 
The outcome event can be desirable (e.g. cure) or undesirable (e.g. an adverse drug reaction). In the later case, it is semantically preferable to refer to relative or absolute risk increase.
 

 

Further Reading:

 
The following articles are taken from the JAMA series "Users' guides to the medical literature."  These articles are available from the Health Sciences Library either in print or online format.
 
Guyatt, G. H., & Rennie, D. (1993). Users' guides to the medical literature. JAMA, 270(17), 2096-7.

Oxman, A. D., Sackett, D. L., & Guyatt, G. H. (1993). Users' guides to the medical literature. I. How to get started. The Evidence-Based Medicine Working Group. JAMA, 270(17), 2093-5.

Guyatt, G. H., Sackett, D. L., & Cook, D. J. (1993). Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA, 270(21), 2598-601.

Guyatt, G. H., Sackett, D. L., & Cook, D. J. (1994). Users' guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA, 271(1), 59-63.

Jaeschke, R., Guyatt, G., & Sackett, D. L. (1994). Users' guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA, 271(5), 389-91.

Jaeschke, R., Guyatt, G. H., & Sackett, D. L. (1994). Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA, 271(9), 703-7.

Levine, M., Walter, S., Lee, H., Haines, T., Holbrook, A., & Moyer, V. (1994). Users' guides to the medical literature. IV. How to use an article about harm. Evidence-Based Medicine Working Group. JAMA, 271(20), 1615-9.

Laupacis, A., Wells, G., Richardson, W. S., & Tugwell, P. (1994). Users' guides to the medical literature. V. How to use an article about prognosis. Evidence-Based Medicine Working Group. JAMA, 272(3), 234-7.

Oxman, A. D., Cook, D. J., & Guyatt, G. H. (1994). Users' guides to the medical literature. VI. How to use an overview. Evidence-Based Medicine Working Group. JAMA, 272(17), 1367-71.

Richardson, W. S., & Detsky, A. S. (1995). Users' guides to the medical literature. VII. How to use a clinical decision analysis. A. Are the results of the study valid? Evidence- Based Medicine Working Group. JAMA, 273(16), 1292-5.

Richardson, W. S., & Detsky, A. S. (1995). Users' guides to the medical literature. VII. How to use a clinical decision analysis. B. What are the results and will they help me in caring for my patients? Evidence Based Medicine Working Group. JAMA, 273(20), 1610-3.

Hayward, R. S., Wilson, M. C., Tunis, S. R., Bass, E. B., & Guyatt, G. (1995). Users' guides to the medical literature. VIII. How to use clinical practice guidelines. A. Are the recommendations valid? The Evidence- Based Medicine Working Group. JAMA, 274(7), 570-4.

Wilson, M. C., Hayward, R. S., Tunis, S. R., Bass, E. B., & Guyatt, G. (1995). Users' guides to the Medical Literature. VIII. How to use clinical practice guidelines. B. what are the recommendations and will they help you in caring for your patients? The Evidence-Based Medicine Working Group. JAMA, 274(20), 1630-2.

Guyatt, G. H., Sackett, D. L., Sinclair, J. C., Hayward, R., Cook, D. J., & Cook, R. J. (1995). Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA, 274(22), 1800-4.

Naylor, C. D., & Guyatt, G. H. (1996). Users' guides to the medical literature. X. How to use an article reporting variations in the outcomes of health services. The Evidence- Based Medicine Working Group. JAMA, 275(7), 554-8.

Naylor, C. D., & Guyatt, G. H. (1996). Users' guides to the medical literature. XI. How to use an article about a clinical utilization review. Evidence-Based Medicine Working Group. JAMA, 275(18), 1435-9.

Guyatt, G. H., Naylor, C. D., Juniper, E., Heyland, D. K., Jaeschke, R., & Cook, D. J. (1997). Users' guides to the medical literature. XII. How to use articles about health-related quality of life. Evidence-Based Medicine Working Group. JAMA, 277(15), 1232-7.

Drummond, M. F., Richardson, W. S., O'Brien, B. J., Levine, M., & Heyland, D. (1997). Users' guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA, 277(19), 1552-7.

O'Brien, B. J., Heyland, D., Richardson, W. S., Levine, M., & Drummond, M. F. (1997). Users' guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA, 277(22), 1802-6.

Dans, A. L., Dans, L. F., Guyatt, G. H., & Richardson, S. (1998). Users' guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient. Evidence-Based Medicine Working Group. JAMA, 279(7), 545-9.

Richardson, W. S., Wilson, M. C., Guyatt, G. H., Cook, D. J., & Nishikawa, J. (1999). Users' guides to the medical literature: XV. How to use an article about disease probability for differential diagnosis. Evidence-Based Medicine Working Group. JAMA, 281(13), 1214-9.

Guyatt, G. H., Sinclair, J., Cook, D. J., & Glasziou, P. (1999). Users' guides to the medical literature: XVI. How to use a treatment recommendation. Evidence-Based Medicine Working Group and the Cochrane Applicability Methods Working Group. JAMA, 281(19), 1836-43.

Barratt, A., Irwig, L., Glasziou, P., Cumming, R. G., Raffle, A., Hicks, N., Gray, J. A., & Guyatt, G. H. (1999). Users' guides to the medical literature: XVII. How to use guidelines and recommendations about screening. Evidence-Based Medicine Working Group. JAMA, 281(21), 2029-34.

Randolph, A. G., Haynes, R. B., Wyatt, J. C., Cook, D. J., & Guyatt, G. H. (1999). Users' Guides to the Medical Literature: XVIII. How to use an article evaluating the clinical impact of a computer-based clinical decision support system. JAMA, 282(1), 67-74.

Bucher, H. C., Guyatt, G. H., Cook, D. J., Holbrook, A., & McAlister, F. A. (1999). Users' guides to the medical literature: XIX. Applying clinical trial results. A. How to use an article measuring the effect of an intervention on surrogate end points. Evidence-Based Medicine Working Group. JAMA, 282(8), 771-8.

McAlister, F. A., Laupacis, A., Wells, G. A., & Sackett, D. L. (1999). Users' Guides to the Medical Literature: XIX. Applying clinical trial results B. Guidelines for determining whether a drug is exerting (more than) a class effect. JAMA, 282(14), 1371-7.

Hunt, D. L., Jaeschke, R., & McKibbon, K. A. (2000). Users' guides to the medical literature: XXI. Using electronic health information resources in evidence-based practice. Evidence-Based Medicine Working Group. JAMA, 283(14), 1875-9.

McAlister, F. A., Straus, S. E., Guyatt, G. H., & Haynes, R. B. (2000). Users' guides to the medical literature: XX. Integrating research evidence with the care of the individual patient. Evidence-Based Medicine Working Group. JAMA, 283(21), 2829-36.

McGinn, T. G., Guyatt, G. H., Wyer, P. C., Naylor, C. D., Stiell, I. G., & Richardson, W. S. (2000). Users' guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA, 284(1), 79-84.

Giacomini, M. K., & Cook, D. J. (2000). Users' guides to the medical literature: XXIII. Qualitative research in health care A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA, 284(3), 357-62.

Giacomini, M. K., & Cook, D. J. (2000). Users' guides to the medical literature: XXIII. Qualitative research in health care B. What are the results and how do they help me care for my patients? Evidence-Based Medicine Working Group. JAMA, 284(4), 478-82.

Richardson, W. S., Wilson, M. C., Williams, J. W. Jr, Moyer, V. A., & Naylor, C. D. (2000). Users' guides to the medical literature: XXIV. How To use an article on the clinical manifestations of disease. JAMA, 284(7), 869-75.

Guyatt, G. H., Haynes, R. B., Jaeschke, R. Z., Cook, D. J., Green, L., Naylor, C. D., Wilson, M. C., & Richardson, W. S. (2000). Users' Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users' Guides to patient care. Evidence- Based Medicine Working Group. JAMA, 284(10), 1290-6.