Module 4: Strategic Reading
Quality Assessment and Appraisal
Now that you have scanned your search results and have amassed those that are relevant to your context and needs, you can settle in to do what we call critical reading. This is where you must plan to read the entire article thoroughly.
Critical Questions to Ask When Reading
Critical reading encourages the reader to think about how an article or argument is constructed, not just what it says or what the study found. Here are some questions to ask yourself when reading:
- What are the central ideas or arguments in the paper?
- Are there words, concepts or research methods used that you don’t understand? If so, ensure you follow up with quality resources to ensure comprehension and extend your learning.
- What is the context for the argument? Have they made explicit connections to existing literature, and located their work within that context?
- Is their literature review clearly outlined and presented in an inclusive manner? Are there any obvious exclusions? If so, are they accounted for?
- Who is the author? Who are they writing for? Have they located themselves and their positionality to the topic at hand? Are there any declared (or perceived) conflicts of interest?
- What evidence is given to support the conclusions?
- Is the evidence credible, that is, does it come from reliable sources?
- Is the logic of the argument sound? What are the steps in the argument that leads from the evidence to the conclusion?
- Are any of their ideas problematic? Are there other lenses that could be used to frame the argument?
Tools for Critical Appraisal
Using a standardized tool for critical appraisal is a good practice to follow. It demonstrates consistency in evaluation of all articles, and when working on a research team, it ensures that each individual is asking the same questions.
There are numerous tools for appraisal to choose from. For the most part, there will be disciplinary norms where certain tools are chosen in order to best represent the specific context. Looking to other reviews in your field to see what tools are employed is a good step.
There are various tools such as checklists designed by research organizations that will help you determine if your source is appropriate. Table 4.1 below outlines a sample of free quality assessment tools available for you to use.
| Tool | Organization | Description |
|---|---|---|
| CASP Checklists | Critical Appraisal Skills Programme (CASP) from the Public Health Resource Unit, NHS, England | Checklists for assessing the reliability, importance, and applicability of studies. Studies include qualitative studies, systematic reviews, randomized control trials and others. |
| Critical Appraisal Tools | The Centre for Evidence-Based Medicine (CEBM) from the University of Oxford | A checklist with questions for assessing the reliability, importance, and applicability of studies based on the research question and inclusion criteria. |
| Risk of Bias 2 (Rob 2) Tool | Cochrane Methods | A manual from the Cochrane Method Network that includes standards for assessing risk of bias in your included studies. |
| Critical Appraisal Tools | Joanna Briggs Institute | Checklists assessing the trustworthiness, relevance and results of different types of studies. |
| Study Quality Assessment Tools | National Heart, Lung and Blood Institute | Quality assessment tools for assessing potential flaws in study methods or implementation of different types of studies. |
To find the tool right for your review check out the Quality Assessment and Risk of Bias Tool Repository from Duke University’s Medical Center Library & Archives.
Key Takeaways
While selecting your sources, you will need to assess the quality of the research question, the methodology used, the validity of the results and whether the study is applicable to your own review. To help you assess the quality of your chosen sources, consult one of the free checklist tools suggested in Table 4.1.
