Advanced Research Skills: Conducting Literature and Systematic Reviews (3rd Edition)

fkovacaj — Tue, 30 Jan 2024 18:11:13 +0000

fkovacaj — Tue, 30 Jan 2024 18:11:20 +0000

fkovacaj — Tue, 30 Jan 2024 18:11:23 +0000

fkovacaj — Tue, 30 Jan 2024 18:11:24 +0000

fkovacaj — Tue, 30 Jan 2024 18:11:25 +0000

fkovacaj — Tue, 30 Jan 2024 18:11:27 +0000

Cover Image

fkovacaj — Tue, 05 Mar 2024 18:19:48 +0000

bies201800206-blkfxd-0001-m

fkovacaj — Wed, 08 May 2024 19:49:01 +0000

jason-goodman-Oalh2MojUuk-unsplash

fkovacaj — Mon, 22 Jul 2024 20:38:04 +0000

desola-lanre-ologun-7d4LREDSPyQ-unsplash

fkovacaj — Mon, 22 Jul 2024 21:02:19 +0000

5051413236_56485456c2_c

fkovacaj — Mon, 22 Jul 2024 21:08:50 +0000

7658288734_de19b54754_c

fkovacaj — Mon, 22 Jul 2024 21:20:46 +0000

5404766001_f6c4b44db4_c

fkovacaj — Tue, 23 Jul 2024 13:37:34 +0000

Screenshot 2024-07-23 094854

fkovacaj — Tue, 23 Jul 2024 13:51:52 +0000

Screenshot 2024-07-23 094854

fkovacaj — Tue, 23 Jul 2024 13:54:10 +0000

Cognitive Behavioural Therapy

fkovacaj — Tue, 23 Jul 2024 14:38:24 +0000

Cognitive Behavioural Therapy (1)

fkovacaj — Tue, 23 Jul 2024 14:40:50 +0000

Cognitive Behavioural Therapy (2)

fkovacaj — Tue, 23 Jul 2024 15:00:48 +0000

Cognitive Behavioural Therapy (3)

fkovacaj — Tue, 23 Jul 2024 15:05:33 +0000

Introduction to Module 1

fkovacaj — Fri, 09 Jul 2021 14:11:07 +0000

Jason Goodman on Unsplash[/caption] In this module, we will go over the different types of academic reviews and a quick overview of the steps involved in conducting them. If you are unsure about which review to choose for your own work this module will help you understand the difference between each review and help you make an informed decision. If you know which review you will be conducting feel free to skip to relevant sections that will answer any questions you might have.

Learning Outcomes

By the end of this module, you will be able to:

Distinguish between the different characteristics of reviews so you can make an informed decision when choosing the appropriate review to meet your scholarly requirements.
Describe the steps involved in conducting your review
Locate existing reviews and analyze how these reviews will apply to your research.

]]>

Workbook Activity 1.1: My Review

fkovacaj — Thu, 30 Sep 2021 22:39:34 +0000

Complete Activity 1.1: “My Review” in your Research Roadmap Workbook. Before you start a review, it’s always a good idea to make sure you take some time to reflect on what your objectives are. This survey will help you articulate what time/resources you have available and what you tentatively plan to undertake. Keep your answers in mind as you read through the module to see if you are on the right track or if you need to modify your objectives. For more information please see Workbook Instructions in the Introduction Module. [caption id="attachment_294" align="aligncenter" width="531"] University student studying. By Flickr user University of Central Arkansa’s Photostream under Creative Commons CC BY-NC-ND 2.0.[/caption]

]]>

Literature vs Systematic Reviews

fkovacaj — Thu, 30 Sep 2021 23:09:23 +0000

Table 1.1. Difference Between Systematic and Literature Reviews Component Literature Review Systematic reviews Definition A literature review is a qualitative integrative summary of published research on a specific topic. The literature review seeks to synthesize what is already known about the topic, and sometimes, explicitly state what is not known, or not well understood. Systematic reviews bring together information from a range of sources to answer a specific research question. They differ from a traditional literature review or narrative review, in that they aim to synthesize and analyze the research in an unbiased, rigorous and systematic way so that it can be used to support evidence-based practice. Goals

Provide summary or overview of topic

Answers a focused [pb_glossary id="162"] clinical [/pb_glossary] question
Eliminate bias

Question

Can be a general topic or a specific question

Clearly defined and answerable clinical question
Recommend using a formula like PICO(T) as a guide

Components

Introduction
Methods
Discussion
Conclusion
Reference list

Literature review section within a research article (*See note below table)

Pre-specified eligibility criteria
Systematic search strategy
Assessment of the validity of finding
Interpretation and presentation of results
Reference list

Number of Authors

One or more

Three or more

Timeline

Weeks to months

Months to years

Requirement

Understanding of topic
Perform searches through databases and other resources

Thorough knowledge of topic
Perform searches of all relevant databases
Statistical analysis resources (for meta-analysis)

Value

Provides summary of literature on the topic

Connects practicing clinicians to high quality evidence
Supports evidence-based practice

Adapted from Kysh, Lynn (2013): Difference between a systematic review and a literature review. Figshare (Poster). Licensed under Creative Commons CC BY 4.0 /order and some words changed.

Note on Table 1.1.

There are two ways to present a literature review: it can be one section in an original study, or it can be a standalone full review. More examples are given in the section on Literature Reviews.

]]>

Literature Reviews

fkovacaj — Fri, 22 Oct 2021 15:47:48 +0000

A literature review is an integrative summary of published research on a specific topic. The literature review seeks to [pb_glossary id="153"]synthesize [/pb_glossary] what is already known about the topic, and sometimes, explicitly state what is not known, or not well understood. [caption id="attachment_291" align="aligncenter" width="510"] Student using laptop. Photo by Desola Lanre-Ologun on Unsplash[/caption] The following are the key characteristics of a literature review:

It contains a clear research question/problem that it is trying to answer.
It is not a list describing or summarizing one piece of literature after another.
It is organized into sections that present themes or identify trends, including relevant theory related to the research question.

Example

A literature review may serve as a stand-alone piece or article such as this article by Francisco Javier Pérez Latre et al., on Social Networks[footnote]Pérez-Latre, F. J., Portilla, I., & Sánchez-Blanco, C. (2011). Social networks, media and audiences: A literature review. Communication & Society, (24)1, 63-74.https://revistas.unav.edu/index.php/communication-and-society/article/view/36221 *Communication & Society is licensed under Creative Commons Attribution-Noncommercial-NoDerivatives 4.0. [/footnote] However, more often a literature review is part of a larger research publication such as this example from Nia Contini et al., on Boating-related fatalities in the Northwest Territories[footnote]Nia Contini, Audrey R. Giles, Gordon Giesbrecht & Tyrone Raddi (2021) The adaptation of the beyond cold water bootcamp course for Inuvialuit communities in Northwest Territories, Canada, International Journal of Circumpolar Health, 80:1, DOI: 10.1080/22423982.2021.1969744[/footnote]

Why Are Literature Reviews Important?

Literature reviews help authors and readers build their knowledge about a specific topic by synthesizing research on the subject. Literature reviews use published research to provide context to the topic and may expose debates within the field, gaps in the research, or provide a summary and analysis of research to date on the topic.

Key Takeaways

Literature reviews are not a summary of a couple of sources you found on your topic. Your review should present themes or identify trends, including relevant theory found in the literature you reviewed.

]]>

Conducting a Literature Review

fkovacaj — Tue, 26 Oct 2021 01:03:40 +0000

Find and examine existing literature reviews.

Formulate a research question.

Search for sources.

Assess the quality of your results and select your sources.

[pb_glossary id="153"]Synthesize[/pb_glossary] the important information from your sources.

Analyze what you’ve found.

Find and Examine Existing Literature Reviews

Prior to starting your own research, you will want to look at existing literature reviews - this is especially important so that you don't duplicate existing work. It can also be helpful to look at the approaches taken for literature reviews similar to your own topic or discipline. Below are some examples of locating existing reviews.

Example

Search in [pb_glossary id="159"]academic journal articles[/pb_glossary] - All published articles have literature reviews. They are a key component of an academic journal article as they create the foundation for new research and establish credibility for the authors. Authors need to demonstrate to peer reviewers and readers that they have a good command of the existing literature, and they have identified a gap they will fill with the present study.
Identify clusters of citations in published articles - Sometimes, literature reviews are clearly labelled as such with a descriptive heading, but more often, they are presented as part of the introduction or background section at the beginning of the article. The telltale sign that you are looking at the literature review is the presence of clusters of citations.
Search for “literature review” articles - While most literature reviews are done in the context of laying the groundwork for an original study, some are published as articles in their own right. When searching for this type of article you can limit part of your search strategy to look for the term "literature review" in the title of the article.

Formulate a Research Question

In general, your research question will tackle the problem you are trying to address by conducting the review. Since constructing a research question can be an in-depth process, we go over it in more detail in Module 2: Formulating a Research Question and Searching for Sources.

Conduct Your Review Using the SALSA Framework

Once you have a research question, there are four stages you can follow when conducting your chosen review. These are known as the SALSA Framework: search, appraisal, synthesis and analysis.

Example

Here is a quick summary of the SALSA steps. [h5p id="1"]

Wait, What happened to the “L” in SALSA?

Did you notice the missing L? We did too! The authors, Grant and Booth (2009) created a simple analytical framework for conducting reviews: Search, Appraisal, Synthesis and Analysis. SASA, however, doesn’t make a memorable acronym, and Academics love a good acronym, so they derived the “L” from the last letter of appraisal: Search, AppraisaL, Synthesis and Analysis (SALSA).

]]>

Systematic Reviews

fkovacaj — Tue, 26 Oct 2021 01:04:14 +0000

What Are Systematic Reviews? A systematic review is also known as evidence synthesis because it brings together information from a range of sources to answer a specific research question. It differs from a traditional literature review, in that it aims to [pb_glossary id="153"]synthesize[/pb_glossary] and [pb_glossary id="158"]analyze[/pb_glossary] the research in an unbiased, rigorous and [pb_glossary id="157"]systematic [/pb_glossary] way so that it can be used to support evidence-based practice.

Characteristics

The scope of the review is established in advance (including the research question and pre-defined eligibility criteria).
A systematic search is conducted in order to identify all studies/resources that would meet the eligibility criteria.
The methodology used to search, assess, analyze and synthesize studies/resources is explicit and reproducible.
The review assesses the validity of the studies/resources for a risk of [pb_glossary id="160"] bias [/pb_glossary].
The review uses explicit methods for extracting and synthesizing study findings ([pb_glossary id="152"]qualitative[/pb_glossary] or [pb_glossary id="161"]quantitative[/pb_glossary]).

Types of Systematic Reviews

There are different types of reviews that involve evidence synthesis and a systematic review is the most well known version. Other examples include a rapid review or a scoping review. We define the different types in more detail below.

Example

[h5p id="2"]

Learning Activity

Determine what type of review would be the best fit for each of the following research questions. [h5p id="3"]

Teams and Time Considerations

A review like a systematic review or a meta-analysis can take at least a year to complete and is usually conducted by a team. If your review is for a class assignment, you can still conduct a systematic review without a team or a year to complete it. If your aim is to eventually publish your review, keep in mind that one of the main goals of a systematic review is to try and eliminate potential bias, and working independently can be viewed negatively. If you must work independently, you should identify this limitation when writing your review. [caption id="attachment_298" align="aligncenter" width="448"] A college girl is studying for her end of year exam. By Flickr user CollegeDegrees360. Licensed under Creative Commons CC BY-SA 2.0[/caption]

Systematic Reviews and Bias

Eliminating bias as much as possible is one of the key characteristics of systematic reviews. By bias, we mean that some type of systematic error has occurred during the review stage that leads to the acceptance of outcomes and conclusions of a study. This can result in the possibility of unfair or misleading information within the reviews. Bias is potentially introduced at any stage of the research process, from formulating your research question to choosing which sources to include. In order to reduce bias in your review, you will need to undertake a quality assessment throughout the review process. Your protocol (see next section) and your [pb_glossary id="163"]screening[/pb_glossary] process will help you reduce your bias. Another way to reduce bias is to work in a team setting, and this is why some reviews require more than one person. We will cover how to check for bias when screening sources in Module 3 Organizing, Managing and Screening Sources.

Why Are Systematic Reviews Important?

A systematic review can generally give us the most dependable answer to a specific research question, and it can identify gaps in our knowledge that require further research. It also communicates the strength of the available evidence and the quality of included studies. This indicates how much confidence practitioners, service users, managers, policy makers, and the popular media should have in the results (Gough & Richardson, 2018).

What Type of Systematic Review is Right for You?

Booth et. al (2016) suggest that your choice of review methods should be determined by five main considerations captured by the acronym, TREAD These and other factors may determine what kind of review is most appropriate to answer your research question.

Learning Activity

Decision Tools The following tools can help you pick the right systematic review:

Knowledge Synthesis Decision Tool Unity Health Toronto Health Science Library, St. Michael’s Hospital
Review Ready Reckoner - Assessment Tool (RRRsAT). Temple University Libraries

Key Takeaways

It is important to understand and meet the specific requirements of your chosen review, especially if you plan to publish your review.

Booth, A., Noye, J., Flemming, K., Gerhardus, A., Wahlster, P., Van Der Wilt, G.J., Mozygemba, K., Refolo, P., Sacchini, D., Tummers, M., Rehfuess, E. (2016). Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions (PDF). Gough, D., & Richardson, M. (2018). Systematic reviews. In Advanced research methods for applied psychology (pp. 63-75). Routledge.]]>

Conducting a Systematic Review

fkovacaj — Thu, 18 Nov 2021 14:02:44 +0000

Figure 1.1 What Authors Do: Systematic Reviews. Designed by Jessica Kaufman, Cochrane Consumers & Communication Review Group, Centre for Health Communication & Participation, La Trobe University, 2011. Licensed under Creative Commons CC BY 4.0.[/caption] [h5p id="4"]

Find Existing Systematic Reviews

Prior to starting your own research, you will want to look at existing systematic reviews - this is especially important so that you don't duplicate existing work. It can also be helpful to look at the approaches taken for systematic reviews similar to your own topic or discipline. You can find existing systematic reviews through a number of ways:

Search published journal articles. Systematic reviews can be published as journal articles. To identify them, add "systematic review" as an additional search term in databases, or look for publication type limits, if available. Here’s an example of a "systematic review" published as a journal article.
Search “Cochrane Database of Systematic Reviews” through your library. This database includes the full text of the regularly updated systematic reviews of the effects of healthcare prepared by The Cochrane Collaboration.
Search different [pb_glossary id="150"]protocol[/pb_glossary] registries. For example, PROSPERO is an international database of prospectively registered systematic reviews in health and social care, welfare, public health, education, crime, justice, and international development, where there is a health related outcome. PROSPERO aims to provide a comprehensive listing of systematic reviews registered at inception to help avoid duplication and reduce opportunity for reporting bias by enabling comparison of completed reviews with what was planned in the protocol.
Search the Campbell Collaboration. The Campbell Collaboration is an international network which publishes high quality systematic reviews of social and economic interventions around the world.

Key Takeaways

Looking at published reviews and protocols can give you an idea of what has already been done and will help you ensure that your own research is original.

Assembling Your Research Team

If you are conducting a systematic review that requires a team these are the typical roles involved:

Reviewers - You may need at least two reviewers working independently to screen abstracts, with a potential third as a tie-breaker
Subject matter experts - Subject matter experts can clarify issues related to the topic,
Statistician - A statistician can help with data analysis
Project leader - A project leader can coordinate and write the final report
Librarians - Librarian(s) can develop comprehensive search strategies and identify appropriate databases

Formulate Your Research Question

Create a Review Protocol

Reviews like a systematic review require a protocol, which is essentially a planning document that indicates how your review will be carried out. Here is a sample protocol template from the Evidence Synthesis Coordinator at the Maritimes Strategy for Patient-Oriented Research (SPOR) Support Unit. This basic form includes all the relevant information needed for a simple protocol. You may wish to register your protocol to avoid the duplication of work and to reduce the potential for bias by enabling a comparison between what was stated in the protocol to the completed review. It is also a way to share your current research interests with the research community at large, and help build your research profile.

Example

How to Register your Protocol: Please see this guide by the National Institute of Health (an agency of the United States government): Systematic Reviews Protocol and Protocol Registries.

Key Takeaways

By creating a protocol you are creating a document that will guide you through the systematic review process. Always refer to it throughout the process to ensure you are on track.

Conducting Your Review Using the SALSA Framework

Once you have a research question, there are four stages you can follow when conducting your chosen review. These are known as the SALSA Framework: search, appraisal, synthesis and analysis.

Example

Here is a quick summary of the SALSA steps. [h5p id="5"]

Wait, What Happened to the “L” in SALSA?

Example

Applying the SALSA Framework to Your Specific Review We’ve provided a quick summary of the framework, and once you have chosen your specific type of review you should consult the following chart by Grant and Booth (2009) for a deep dive into each stage of the SALSA framework for your specific review.

A typology of reviews: an analysis of 14 review types and associated methodologies. (Grant & Booth, 2009)

PRISMA: The Systematic Review Checklist

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA is the recognized standard for reporting evidence in systematic reviews and meta-analyses. The standards are endorsed by organizations and journals in the health sciences. It allows other researchers to assess strengths and weaknesses of the review and assists with future replication of the review methods. The 2020 PRISMA statement consists of a 27-item checklist and a 4-phase flow diagram.

Example

For more information, consult the PRISMA Explanation and Elaboration document. If you are conducting a scoping review, see The PRISMA-ScR (PRISMA Extension for Scoping Reviews)

Key Takeaways

A large amount of time and resources go into conducting a systematic review. To make sure you are ready to carry out a review, use the Knowledge Synthesis Readiness Checklist from Unity Health Toronto.

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health information & Libraries Journal, 26(2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x]]>

Key Takeaways

fkovacaj — Fri, 22 Oct 2021 17:04:11 +0000

Literature Reviews

A literature review is a qualitative integrative summary of published research on a specific topic.
- It is not a list describing or summarizing one piece of literature after another.
- It is organized into sections that present themes or identify trends, including relevant. theory related to the research question.
One way to conduct reviews is with the SALSA Framework: search, appraisal, synthesis and analysis.

Systematic Reviews

A systematic review brings together information from a range of sources to answer a specific research question. It differs from a traditional literature review, in that it aims to synthesize and analyze the research in an unbiased, rigorous and systematic way so that it can be used to support evidence-based practice.
Some reviews need to be conducted in a team and can take up to 1 year to complete.
Systematic reviews usually have a research protocol
One way to conduct reviews is with the SALSA Framework: search, appraisal, synthesis and analysis.
PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyse

]]>

Systematic Reviews

Helpful Websites

Systematic Reviews (Toronto Metropolitan University)
Cochrane Database of Systematic Reviews
Systematic Review Guide (Unity Health Toronto)

Registering Your Systematic Review

Articles and Books

Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19-32.

Boland, A., Cherry, M. G., & Dickson, R. (2014). Doing a systematic review: A student's guide. Sage

SAGE.

Ganann, R., Ciliska, D., & Thomas, H. (2010). Expediting systematic reviews: Methods and implications of rapid reviews. Implementation Science, 5(1), 56-56.

Glass, G. (1976). Primary, Secondary, and Meta-Analysis of Research. Educational Researcher, 5(10), 3-8.

Gough, D., & Richardson, M. (2018). Systematic reviews. In Advanced research methods for applied psychology (pp. 63-75). Routledge.

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health information & Libraries Journal, 26(2), 91-108.

Higgins J.P.T, Thomas J, Chandler J, Cumpston M, Li T, Page M.J, Welch V.A. (Eds). (2023). Cochrane Handbook for Systematic Reviews of Interventions version 6.4. Cochrane.

]]>

Introduction to Module 2

fkovacaj — Thu, 09 Dec 2021 20:57:42 +0000

SFU Continuing Studies students at work in the classroom. By Flickr user Simon Fraser University – Communications & Marketing under Creative Commons CC BY-ND 2.0[/caption] In this module we go over the framework for searching the literature. Keep in mind that searching is a non-linear and potentially iterative process; as such you may need to review and revise earlier steps at any point. The steps to searching the literature are:

Formulating a research question
Identify search terms (keywords)
Brainstorm more search terms
Construct a search strategy
Select and Search Databases
Manage your search strategies and results (Covered in Module 3: Organizing, Managing and Screening Sources)

We will show you how to use special techniques like forward and backward citation searching to find more sources. Finally, if you are conducting a systematic review you should check out our special section on how to document your search strategy and results.

Learning Outcomes

By the end of this module, you will be able to:

Determine your research question
Design and refine your own search strategy
Choose appropriate resources to search
Know how to use other serendipitous methods of information gathering and when to stop searching
Document your search strategy
Consider the use of emerging technologies in the review process
Recognize and acknowledge the role of libraries and librarians in the review process

]]>

Activity: How Are Your Search Skills?

fkovacaj — Thu, 09 Dec 2021 21:20:31 +0000

Learning Activity

Test your knowledge of Boolean, truncation and other searching techniques. [h5p id="7"]

Need a Refresher on Basic Searching Skills?

Check out these guides:

Searching for Sources (Research Skills Tutorial). Toronto Metropolitan University Library
Library Research Skills Tutorial. McMaster University Library

]]>

Formulating a Research Question

fkovacaj — Thu, 09 Dec 2021 22:12:17 +0000

Module 1: Types of Reviews, conducting a “pre-search” is a crucial first step in devising the research question. A well-formulated research question informs the research process. It can focus your information needs (i.e. identify inclusion and exclusion criteria), help to identify key search concepts, and guide you in the direction of relevant resources.

Types of Research Questions

There are two general types of research questions: [pb_glossary id="161"]quantitative[/pb_glossary] and [pb_glossary id="152"]qualitative[/pb_glossary].

Quantitative Research Questions

Types of quantitative questions can be categorized as explanatory (i.e., relationship-based), descriptive, or comparative.

Explanatory questions aim to discover cause-and-effect relationships by comparing two or more variables, individuals or groups based on differing outcomes.
Descriptive questions will often quantify a single variable but may include multiple variables within a question. They typically ask for measurements, and can begin with: “how much”, “what percentage”, “how frequently”, or “how many”.
Comparative questions are designed to identify the “difference between” a dependent variable and two or more groups. These questions tend to begin with “what is the difference” or “what are the differences”.

Qualitative Research Questions

Qualitative questions aim to discover meaning or gain an understanding of a phenomenon. They ask questions that cannot be measured with specific numbers and statistics. Qualitative research questions often contain words like [pb_glossary id="164"]"lived experience"[/pb_glossary], "personal experience", "understanding", "meaning", and "stories".

A Comparison of Quantitative and Qualitative Questions

So how do quantitative and qualitative research questions differ when you are conducting a search? In Table 2.1 below, we provide some examples of research topics. Each topic can either be used for a quantitative or qualitative research question. For each question, the category of research question is clarified.

Table 2.1. Examples of Quantitative and Qualitative Research Questions
Topic	Quantitative Research Questions	Qualitative Research Questions
Housing insecurity for children	Explanatory: How does experiencing housing-insecurity in childhood impact a child’s relationships in adulthood?	Lived experience: What are the lived experiences of children experiencing housing-insecurity while attending elementary School?
Nursing and workplace violence	Descriptive: What percentage of nurses experience violence in the workplace on night shifts?	Personal understanding: How do nurses conceptualize their role in mitigating workplace violence?
Mental health of Canadian immigrants	Comparative: What are the differences in perceptions towards accessing mental health services between immigrants and their Canadian-born children?	Personal experience: How do newly arrived Canadians experience seeking help for mental health concerns?

Key Takeaways

Regardless of the type of question being asked, a good research question cannot be answered with a simple yes or no (as demonstrated by the sample questions in Table 2.1).

]]>

Identify Search Terms (Keywords)

fkovacaj — Thu, 09 Dec 2021 22:55:18 +0000

[pb_glossary id="165"]keywords[/pb_glossary], (also known as search terms) is important for effective literature searching. Your search terms are terms that will appear somewhere within the resource (e.g. title, abstract, or author keywords). You can identify search terms from your research question by highlighting, underlining or circling the main ideas that must appear in the article. Your search concepts are the most important words in your research question.

Example

What are the keywords you identify within this research question? "What is the relationship between flexible work schedules and staff retention?" [h5p id="12"]

Keywords are easier to identify once the research question has been formulated using one of the stated frameworks (e.g. PICO or PS) listed in Table 2.2 of Formulating a Research Question. Below is an example of how to use the PICO(T) framework to identify search concepts for a specific research question.

Example

Say you have the following research question: How effective is cognitive behavioural therapy in improving mild-to-moderate depression in adolescents? Let’s break down the formula components by concept.

Formula component	Concept
Patient, Population or Problem (P)	Adolescents with mild-to-moderate depression
Intervention (I)	Cognitive behavioural therapy
Comparison, Control Intervention (C)	There is no concept here
Outcome (O)	Symptom reduction

Formula component

Concept

Patient, Population or Problem (P)

Adolescents with mild-to-moderate depression

Intervention (I)

Cognitive behavioural therapy

Comparison, Control Intervention (C)

There is no concept here

Outcome (O)

Symptom reduction

Be aware that not all questions will include each of the framework’s components. For example, the above question does not have a comparison, so there is no Comparison/Control Intervention (C) concept. Also, many experts advise that when conducting a systematic review, you should be careful about specifying a search term for “outcome.” It can result in considerably lower results and produce bias in your study (Frandsen, Nielsen, Lindhardt, and Eriksen, 2020).

Learning Activity

Identify Search Concepts Drag and drop the correct search terms using PICO (Patient/Population/Problem, Intervention, Comparison/Control, Outcome) for the following research question: [h5p id="13"]

Frandsen, T. F., Nielsen, M. F. B., Lindhardt, C. L., & Eriksen, M. B. (2020). Using the full PICO model as a search tool for systematic reviews resulted in lower recall for some PICO elements. Journal of Clinical Epidemiology, 127, 69-75.]]>

Related Keywords

fkovacaj — Thu, 09 Dec 2021 23:18:57 +0000

main keywords, similar or related keywords and controlled terms (also called subject headings). Searching by a keyword will retrieve resources where the author(s) used that specific term. For this specific reason, you should also brainstorm similar or related keywords to incorporate into your search.

Methods for Identifying More Keywords

You can identify more keywords in multiple ways. Below are two methods.

Method 1: Use a Concept Model or Map

To use this method:

Write your research topic or question, along with any ideas and concepts associated with it on a blank sheet of paper.
Use themes to group your ideas, and connect related concepts using lines.
Remember to include the following:
- Synonyms
- Acronyms
- Brand names and generic names
- Variation in spelling (e.g. "paediatric" or "pediatric")

Please see Figure 2.1 below for an example where your research question is “How effective is cognitive behavioral therapy in improving mild-to-moderate depression in adolescents?” [caption id="attachment_61" align="aligncenter" width="624"] Figure 2.1. Example of a “Concept Map” to generate more keywords. Source: Jo-Ann Petropoules. Licensed under Creative Commons CC BY 4.0[/caption]

[h5p id="14"]

Method 2: Use Your Main Keywords in a Database

To use this method:

Locate an article on your topic.
Scan the title, abstract, and author keywords to identify more keywords (see Figure 2.2 below) to use in your search.

[caption id="attachment_306" align="aligncenter" width="704"] Figure 2.2. Example of how to find keywords in a journal article. From Stanzani-Maserati, M., De Matteis, M., Bosco, L., Baccari, F., Zenesini, C., Mitolo, M., La Morgia, C., Gallassi, R., & Capellari, S. (2024). "Build Your Village"-Conducting the Village Test on Cognitively Impaired Patients: A First Journey into Alzheimerland. Brain sciences, 14(6), 523. https://doi.org/10.3390/brainsci14060523. Licensed under a Creative Commons CC BY 4.0.[/caption]

Key Takeaways

Some databases like Ovid MEDLINE, PubMed and EBSCO CINAHL use controlled vocabularies like MeSH as well as keywords. The next section will explain what controlled vocabularies are and how to use them. If you know you are going to use a database with controlled vocabularies, please check out the next section.

Learning Activity

Brainstorm more concepts [h5p id="15"]

]]>

Cognitive Behavioural Therapy (4)

fkovacaj — Tue, 23 Jul 2024 15:07:09 +0000

Screenshot 2024-07-23 114527

fkovacaj — Tue, 23 Jul 2024 15:46:25 +0000

Screenshot 2024-07-23 115730

fkovacaj — Tue, 23 Jul 2024 16:05:57 +0000

6262379915_7a4827efcb_c

fkovacaj — Tue, 23 Jul 2024 16:17:57 +0000

39426563864_11b3ed85d6_c

fkovacaj — Tue, 23 Jul 2024 16:40:27 +0000

5405371556_cf63a559e1_c

fkovacaj — Tue, 23 Jul 2024 17:06:14 +0000

Formulas for Constructing Research Questions

fkovacaj — Sun, 09 Jan 2022 19:34:40 +0000

Formulas Using a formula is another way to construct a research question and is recommended if you are conducting a systematic review. There are various formulas you can use to craft your question, see Table 2.2 below.

**Table 2.2. Formulas for Constructing Research Questions**
Formula	Components Within the Formula	Example
PICO(T)	Population/Problem Intervention/Exposure Comparator Outcome Time Period/Type of Study Note: the time period and type of study are optional	P: Patients who have undergone knee surgery I: Post-operative infection C: Patients without post-operative infection O: Duration of recovery
PS	Population/Problem Situation	P: New immigrants S: Accessing mental Health Care
PIE	Population Intervention Effect / Outcome	P: Children experiencing homelessness I: Breakfast program at school E: Academic performance
PEO	Population/Problem Exposure Outcomes/Themes	P: Health professionals E: Caring for patients with Dementia O: Attitudes
PCC	Population Concept Context	P: Pediatric patients with sleep disturbance C: Quality of life C: After tonsillectomy surgery
SPICE	Setting Perspective Intervention Comparison Evaluation	S: Pediatric hospital rooms P: Pediatric patients I: Therapy dog visits C: No therapy dog visits E: Reduced anxiety
SPIDER	Sample Phenomena of Interest Design Evaluation Research type	S: Young parents P of I: Early Literacy programs D: Survey E: Experiences R: Qualitative

These frameworks aid in identifying the important parts (i.e., concepts) that can be used in formulating a research question. Below are two examples using the PS and PEO frameworks.

Example

Can you think of a research question using these identified PS concepts? Population/Problem: Family members’ of dying loved ones Situation: Placing them in palliative care [h5p id="8"]

Example

Can you think of a research question using these identified PEO concepts? Population/Problem: Caregivers of family members in palliative care Exposure: Psycho-educational group intervention Outcomes/Themes: Improved quality of life [h5p id="9"]

Is My Question Too Narrow or Too Broad?

When trying to settle on a research question, ensure you are choosing something of interest, but not so narrow that you are unlikely to uncover any information, but not too broad so that you are overwhelmed with content and don’t know where to start. Below are examples of too broad and too narrow questions and an explanation to help understand why.

Example

Why is this example too broad a question? "A systematic review of the literature on palliative care interventions." [h5p id="10"]

Example

Why is this example too narrow a question? "Effectiveness of early reading intervention programs for children aged 5-7 in the private school system in East Toronto." [h5p id="11"]

A pre-search will assist you with this step. If you need a refresher on conducting a pre-search, see Module 1: Types of Reviews. Once you have a good understanding of the existing literature, you will be able to confidently formulate your research question.

Key Takeaways

Keep in mind that your question may change over time as you delve deeper into the literature. This is a normal course of events. You are still in the outline stage! Part of the process is refining and leading to sharper focus as you learn. For guidance on developing the research question see the further reading section.

]]>

Controlled Vocabularies

fkovacaj — Wed, 19 Jan 2022 20:44:52 +0000

Controlled vocabularies are a standardized set of terms used to describe the content of a resource in a database. This is known as [pb_glossary id="166"]indexing[/pb_glossary]. Using controlled vocabulary terms will usually generate fewer and more relevant results. However, you must know the exact term/vocabulary to use. An example is provided below.

Example

Articles on cancer research can include terms like: tumour, cancer, or neoplasm. In Ovid MEDLINE, the controlled vocabulary for these terms is Neoplasms. By searching for Neoplasms you should retrieve all the articles where the author(s) used the terms tumour, cancer, or neoplasm (as long as they have been properly indexed).

When to Use Controlled Vocabularies

The following are situations in which you would use controlled vocabularies:

If your keyword searching is giving you too many or too few results, try using controlled vocabulary terms.
If you are doing a comprehensive search (e.g., a systematic review), you should combine both keywords and controlled vocabulary terms to increase the number of resources retrieved.

How to Use Controlled Vocabularies

Search strategies must be customized for each database for a number of reasons, one of which is that they use different controlled vocabularies. The tutorials in Table 2.3 below cover how to use controlled vocabularies in specific databases. Review these tutorials if you’d like to learn more.

Table 2.3. Databases and Controlled Vocabulary Tutorials
Database	Controlled Vocabulary	Tutorial
Ovid MEDLINE and PubMed	MeSH	MeSH tutorial
APA PsycInfo	APA Thesaurus	APA PsycInfo Tutorial
EBSCO CINAHL	CINAHL Subject Headings	CINAHL Tutorial
Ovid Embase	Emtree	Embase Tutorial (PDF)

Key Takeaways

Databases like GEOBASE or Social Sciences Abstracts do not automatically identify relevant controlled vocabularies. Instead, you need to search the database’s thesaurus or browse their list of controlled terms. Consult the database’s help page, or contact your library for assistance.

How to Find More Controlled Vocabulary

Below are the steps to follow when finding more controlled vocabularies:

Search for each keyword in a database separately. This will allow you to identify additional controlled vocabulary. For example, the database PubMed allows you to search for MeSH headings one at a time.
- The CADTH Search Filters Database can also help you look up the MeSH heading across databases such as Scopus, CINAHL, PubMed, PsycInfo, MEDLINE and Embase.
Find an article on your topic and look at the controlled vocabulary assigned to it. For example, in PubMed, look for the heading “MeSH” (the name for the controlled vocabulary in PubMed) under the heading of “Cited by.” See Figure 2.3 below for an example.
In other databases, such as Social Sciences Abstracts, find an article on your topic and look for the heading “Subjects” to help identify relevant controlled vocabularies.

[caption id="attachment_322" align="aligncenter" width="943"] Figure 2.3. MeSH terms in an article in PubMed. From Vittorini, M. G., Sahin, A., Trojan, A., Yusifli, S., Alashvili, T., Bonifácio, G. V., Paposhvili, K., Tischler, V., Lampl, C., Sacco, S., & School of Advanced Studies of the European Headache Federation (EHF-SAS) (2024). The glymphatic system in migraine and other headaches. The journal of headache and pain, 25(1), 34.[/caption]

Key Takeaways

Not all search terms will map to an appropriate subject heading. If this is the case, search for it as a keyword. This option is available after the list of subject headings in most databases.

]]>

Construct a Search Strategy

fkovacaj — Sun, 09 Jan 2022 20:20:35 +0000

Boolean Operators Boolean operators connect your search concepts and search terms (i.e., controlled vocabulary and keywords) together. The three basic operators are: OR, AND, and NOT. An example is shown below, followed by a deeper dive into each of these operators.

Example

(Cognitive Behavioral Therapy OR CBT) AND Depression

The OR Operator

The OR operator broadens your search results by retrieving records that contain either or both of your search terms or concepts. Usually, terms within a concept are combined using the Boolean OR operator. In the Figure 2.4 below, the database will retrieve records that contain either or both of the terms Cognitive Behavioural Therapy OR CBT. [caption id="attachment_316" align="aligncenter" width="697"] Figure 2.4. Example of the “OR” operator in Boolean.[/caption]

The AND Operator

The AND operator narrows the search results by retrieving records that contain both of your search terms or concepts. Usually, different concepts are combined using the Boolean AND operator. In the Figure 2.5 below, the database will retrieve sources that contain Cognitive Behavioural Therapy AND Depression [caption id="attachment_317" align="aligncenter" width="697"] Figure 2.5. Example of the “AND” operator in Boolean.[/caption]

NOT

The NOT operator narrows the search results by eliminating a specific search term. You should use the NOT operator with caution. This is because it can eliminate relevant results that happen to mention the term you are "noting" out. In Figure 2.6 below, any articles that mention both adult and adolescent will be excluded from the result set. [caption id="attachment_320" align="aligncenter" width="697"] Figure 2.6. Example of the “NOT” operator in Boolean.[/caption]

Key Takeaways

Depending on the search engine (e.g., Google Scholar) or database (e.g., PubMed), the operator(s) may be required to be entered in upper case letters. Consult the search engine’s or database’s Help information or Search Tips for details about how to enter the Boolean Operators.

Learning Activity

Test your Boolean Knowledge [h5p id="16"]

Using Truncation and Wildcards

[pb_glossary id="168"]Truncation[/pb_glossary] and [pb_glossary id="169"]wildcards[/pb_glossary] can be applied to a keyword search to broaden your results and allow you to look for variations of words.

Truncation

You can use the truncation symbol to avoid typing out all possible variations of a word. For example, surg* will retrieve surgery, surgeries, surgeon, or surgical. The truncation symbol should be used with caution to ensure relevant words are being retrieved. The most common symbol used is an asterisk (*). The asterisk symbol is used in the following platforms and databases:

EBSCO (e.g., CINAHL, Business Source Complete)
Ovid (e.g., MEDLINE, Embase, PsycInfo)
ProQuest
ERI,
PubMed
Web of Science

Wildcard Symbols

Wildcard symbols can be used to substitute for one or more characters, or any single character. This is useful when dealing with variant spelling (e.g., pediatric and paediatric). One of the most common symbols used is the question mark (?). For example, in the Ovid database, p?ediatric* will retrieve pediatric OR paediatric OR pediatrics OR paediatrics. Databases use different wildcard symbols, so check the database Help information or Search Tips for details about which symbol to use. ]]>

Select Resources and Search

fkovacaj — Sat, 15 Jan 2022 15:30:51 +0000

For systematic reviews, searching multiple databases is essential for reducing [pb_glossary id="160"]bias[/pb_glossary] in your results. Only using one database like PubMed means you have ignored relevant studies found in other databases like Embase.

Subject Databases

Subject databases are accessible through your academic library and contain published material belonging or pertaining to a specific discipline or multiple disciplines, e.g.,health sciences, business, social work, or law. Many subject databases are hosted on the platforms EBSCO and Proquest for example. These search platforms allow users to search and retrieve content, and may look and work similarly to each other. The type of material in different subject databases depends on the subject, but most will include academic peer-reviewed articles, newspaper and magazine articles, and reports. To find a subject specific database, go to your library’s website and look for a research guide for your discipline. This guide will include a list of article databases for your discipline, and the descriptions below them will help you select among them. Alternatively, look for a list of databases by subject on your library website, and choose an article database which covers your subject. If you have questions about which database to choose, contact your librarian for help.

Key Takeaways

There are also free options if you are not affiliated with an academic library. Public libraries have some databases you may search with a library card. Also, there are some free databases online such as PubMed (health sciences subjects), DOAJ (general subjects), Dimensions (general subjects) and Google Scholar (general subjects). Further free (Open Access) resources are listed on this Open Access Educational Resources Guide.

How to Search in Subject Databases

In general, all subject databases will have a basic search box and an advanced search box. We recommend you use the advanced search box, as it provides more options to optimize your search. The advanced search box will often allow you to choose where you want your keywords to appear within the resource (e.g., in the title or the abstract and body or anywhere). An example of an advanced search box is found in Figure 2.7 below. [caption id="attachment_68" align="aligncenter" width="928"] Figure 2.7. Example of an advanced search box in PubMed. This screen grab, from PubMed National Library of Medicine (NLM) is included on the basis of fair dealing.[/caption] Limiters are found in all subject databases and they allow you to limit your search by date, type of source, language, and in some databases by population and methodology (see Figure 2.8 below). This is particularly useful if you are conducting a systematic review, as the limiters will help with your exclusion criteria. [caption id="attachment_324" align="aligncenter" width="1024"] Figure 2.8. Example of date range limits in PubMed. This screen grab, from PubMed National Library of Medicine (NLM) is included on the basis of fair dealing. [/caption] Once you have chosen your subject databases, take a note if they are owned by a particular platform (i.e., EBSCO, ProQuest, Ovid, etc). Each of these platforms will have tutorials on how to conduct an advanced search. Below are video tutorials from some of the major subject database platforms.

Example

Advanced Searching Tutorial Videos:

Google Scholar

Google Scholar searches articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites. The majority of sources will be behind pay-walls and you will not be able to access them. However, you can optimize Google Scholar to search your academic library’s collection, at which point you will be able to access any item your library owns. For a list of the pros and cons of using Google Scholar please see Table 2.6.

Table 2.6. The Pros and Cons of Using Google Scholar
Pros	Cons
Familiar Google interface - easy to search Very broad coverage Integrated citation information and author profiles Results can be linked to your library collection	Limited search capabilities and very large result sets make precision searching impossible, e.g.,: No subject headings Limited sorting and filtering options Algorithm-based (takes the control out of the searcher's hands) Not clear how "scholarly" is defined (leads to some questionable results) Lacks the focus and quality control present in other scholarly databases

From McMaster University, Health Science Library, Google Scholar. Licensed under Creative Commons CC BY-NC-SA 4.0.

How to Search Google Scholar

The following video demonstrates how you can search Google Scholar.

[embed]https://www.youtube.com/embed/xPsJdhSZLRY[/embed]

UTS Library, Google Scholar: Advanced Searching [2:53]. Licensed under a Creative Commons CC BY 4.0. Closed Captions

Grey Literature and Other Sources

Grey literature refers to any information that is not commercially published, and can include a wide range of material types, such as:

Theses and [pb_glossary id="171"]dissertations[/pb_glossary], e.g., databases such as Proquest Dissertations or Theses Canada
Government documents and websites
Clinical trials registries
Conference proceedings, e.g., databases such as Scopus or Web of Science include citations to conference proceedings
Research reports

How to Search Grey Literature

Knowing where to search for grey literature can be challenging. There are a variety of suggestions in the following library guide from Simon Fraser University: Grey literature: What it is & how to find it. A good approach is to be strategic and consider where appropriate grey sources might be located. For example, if you know of any government agencies, non-profits, professional associations, research institutes, and other organisations that may be producing content relevant to your topic, you may wish to search their website for any publications and reports. At a very minimum, looking at graduate theses and dissertations and conference proceedings ( often sites of emerging research) should be a component of your grey literature search. Searching for grey literature will likely involve the use of a search engine such as Google, so it will be important for you to critically evaluate sources before including them in your research. For some tips on avoiding fake news and the spread of misinformation, review the following guide from Toronto Metropolitan University Library: How to Identify Fake News.

Replicating Your Search Strategy Across Different Databases

Your search strategy will need to be "translated" for use in multiple databases and you will have to look out for the following:

Controlled vocabulary can vary across databases, e.g., MEDLINE uses MeSH while Embase uses Emtree
Search options, limiting options, and availability of search fields will differ between databases

Translating Search Strategies for Systematic Reviews

Systematic reviews require you to use the same search strategy across different databases. However, as mentioned above, different subject databases might not allow you to use the exact same search. Consider the steps below on how to translate your search strategy from one database to another (McMaster University Library, 2024:

Open the database in which you created your first (original) search strategy, e.g., Ovid MEDLINE. This will serve as a guide to translating your strategy into the other database(s).
Open the database you wish to translate your search into, e.g., Ovid Embase, CINAHL, Ovid PsycINFO.
In the search box of the new database, e.g., Ovid Embase, enter the first term or phrase from your original strategy, e.g., Ovid MEDLINE.
Identify and search on the relevant controlled vocabulary. Here are a few notes:
- Remember, controlled vocabulary vary across databases
- An equivalent controlled vocabulary may not exist across databases. In such situations, it is recommended that you search it as a keyword.
- For all keyword or phrase searches, check to see if there is a controlled vocabulary in the new database, regardless of whether the equivalent exists in the original search strategy.
Repeat step 4 for subsequent terms or phrases from the original search.
Remember that searching is an iterative process. Revise your initial search strategy if you find a unique term in subsequent databases.

McMaster University Library. (2024, June 6). Translate a Search Strategy. How to Search the Literature (Advanced).]]>

Using Citations to Find More Sources

fkovacaj — Sat, 15 Jan 2022 16:07:07 +0000

consult the citations within an article. By consulting the citations you can find relevant titles that you might have missed in your search. The advantage of this method is that you can find sources about a subject quickly and relatively easily. The disadvantage of this method is that you are searching backward, or retrospectively, so each source you find will be older than the previous one. This is why you need to use a forward citation searching method in combination with your backward citation search. In the forward searching method, researchers use tools within databases like Google Scholar and Web of Science to see who has recently cited your source. This way you can find new and relevant sources.

Method 1: Backward Searching

In this method, review the references used in key articles. For example, search through the article for the "References" section as in Figure 2.9 below. [caption id="attachment_72" align="aligncenter" width="1024"] Figure 2.9. Example of backward citation searching in PubMed by looking at the “References” section of an article. This screen grab from PubMed, National Library of Medicine (NLM) is included on the basis of fair dealing.[/caption]

Method 2: Forward Searching

In this method, see what articles have cited your key articles. For example, search through the "Cited by" section of the article as in Figure 2.10 below. [caption id="attachment_72" align="aligncenter" width="1024"] Figure 2.10. Example of forward citation searching in PubMed. This screen grab from PubMed , National Library of Medicine (NLM) is included on the basis of fair dealing.[/caption] Many databases will include live links to an article’s references, as well as live links to other articles that have cited that article. The following video demonstrates how to do citation searching in Google Scholar:

[embed]https://www.youtube.com/embed/u8Sn3bnT5q8[/embed]

CSUDH Library. Citation Tracking. Licenced under Creative Commons CC BY 4.0. Closed Captioned

Citation Mapping Tools

If you’ve ever clicked on a link titled something like “suggested for you” or “you may also like” on your favourite online store, you were taking advantage of an algorithm that uses your interests to identify related items. Algorithms search for [pb_glossary id="228"]metadata[/pb_glossary] to collate similar items. This is similar to clicking a hashtag on social media sites like Instagram. Citation mapping tools are an academic version of this process. Not only do these tools look for items that have similar metadata, they also leverage the world of bibliometric data to create a visual representation of the world that surrounds a single article. These tools require you to enter a citation (often referred to as a “seed paper”) you are interested in further exploring. The results will vary depending on the tool you select, but in general, you will be led to further works related to the seed paper either by the similarity in their bibliographies or adjacency to similarly cited works.

When should you use a citation mapping tool?

It may be very tempting to want to rely on a tool that works quickly, and uncovers potentially new articles. However, be cautious when deciding how to make use of a citation mapping tool. The majority of these tools are using algorithms that are not transparent in their composition and operation. This is problematic if you are engaged in a systematic review where replicability of the search is essential to the methodology. Moreover, over-reliance on AI tools when engaging in an academic exercise is a slippery slope. One must always keep in mind that the entire purpose of a literature review is to gain a deeper understanding of the existing literature and thought in your field. AI tools can be helpful to enhance productivity; they are not a replacement for critical or in-depth thinking about information. Secondly, there are existing tools and methods that are thoroughly documented and have been used for decades. The novelty of an AI tool should not replace the use of existing approaches to forward and backward citation searching (also known as citation chasing) that can be replicated, and are widely acknowledged as standard in evidence synthesis. The old adage of “if it ain’t broke, don’t fix it” is applicable here. Using traditional citation indexes such as Scopus or Web of Science should be sufficient for reliable citation chasing. Finally, it can be very tempting to use a tool that you feel will save you time. However, research should not feel like a race, and skimming your way through a literature review is a mistake. The devil, they say, is in the details. Poor reading habits such as over-reliance on abstracts or machine-generated summaries can lead to misreadings of studies in your review. Using an abstract or summary to determine relevance is appropriate; relying solely on them for interpretation of a studies’ purpose, methods, findings and conclusions is a recipe for misinterpretation. This can snowball and lead to embarrassing errors, and worse, charges of dishonesty. [caption id="attachment_235" align="aligncenter" width="587"] Image source: França, T. F., & Monserrat, J. M. (2019). To read more papers, or to read papers better? A crucial point for the reproducibility crisis. BioEssays, 41(1), 1800206.[/caption] If, however, you are not engaged in a proper systematic review or meta-synthesis, and instead are working in the area of qualitative evidence synthesis (QES), or are engaged in a wide-ranging scoping search, you may want to consider using a citation mapping tool to expand the pool of sources as a supplementary search method. According to the Cochrane Handbook for Systematic Reviews of Interventions, “poor indexing of qualitative studies makes citation searching (forward and backward) and the Related Articles features of electronic databases particularly useful”(Harris et al., 2018). Furthermore, methodological studies have found that supplementary search methods (i.e. those methods undertaken outside of database searching), including citation chasing, is beneficial in identifying unique studies, especially in the social sciences (Cooper et al., 2017). In short, using a citation mapping tool for discovery when supplementary search strategies are called for can be immensely useful. As a tool to enhance productivity, however, it should be given careful thought and not be solely relied upon, particularly given the mysterious nature of its functionality.

How to select a citation mapping tool?

It is challenging to recommend any one tool. Much will depend on your discipline, the goal of your research, or even just personal preference. Many tools are open access, others require users to create a user account, and still others are subscription based. Aaron Tay, a librarian from Singapore Management University, maintains a comprehensive list of currently available citation mapping tools on his blog. The best way to determine which tool is right for you is to go ahead and experiment with them! Select an article with which you are very familiar and use it as your “seed paper” in the tools you are keen to try.

Cooper, C., Booth, A., Britten, N., & Garside, R. (2017). A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: A methodological review. Systematic Reviews, 6(1), 234. Harris, J. L., Booth, A., Cargo, M., Hannes, K., Harden, A., Flemming, K., Garside, R., Pantoja, T., Thomas, J., & Noyes, J. (2018). Cochrane Qualitative and Implementation Methods Group guidance series—paper 2: Methods for question formulation, searching, and protocol development for qualitative evidence synthesis. Journal of Clinical Epidemiology, 97, 39–48.]]>

When to Stop Searching

fkovacaj — Sat, 15 Jan 2022 16:29:01 +0000

Tips on When to Stop Searching

1. Determine that your search strategy is working. You’ll know it is working when your search results include major studies/sources that have been recommended or, after a quick analysis of a few articles, you can tell they meet your criteria.
2. You have searched all relevant databases.
3. You used the citation searching techniques (See previous section). When using these methods you should see studies that you’ve already found in your database search and some new ones.
4. You decided to add in some new search terms to a database search strategy and it yielded no new sources.
5. You decided to remove a search term and the major studies you’ve found in previous searches have disappeared on the results page.

]]>

Documenting Your Search Strategy and Results for Systematic Reviews

fkovacaj — Sat, 15 Jan 2022 16:33:36 +0000

keep track of:

Each database and platform (PsycINFO, PubMed, Medline etc) searched
Date each search was conducted
Subject headings and keywords used (including whether terms were exploded, truncated, etc)
Number of results retrieved for each search within each database
Total number of records over all
Number of duplicates identified
Numbers pre-screening and post-screening

Documentation Templates

In Module 1 we learned that PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. The PRISMA Group has developed an extension to PRISMA to assist researchers in documenting their literature searches for systematic reviews. The checklist identifies what to document in terms of information sources and methods, search strategies, peer review, and records management. The PRISMA flow diagram demonstrates how to document your final numbers.

Example

Take a moment to review the checklist and the flowchart and save a copy for your own review.

PRISMA Checklist (docx)

]]>

Workbook Activity 2.1: Create Your Own Search Strategy

fkovacaj — Sat, 15 Jan 2022 16:38:36 +0000

Activity 2.1: “Create Your Own Search Strategy” in your Research Roadmap Workbook This exercise allows you to document your search strategy. Don’t worry if you are still trying to figure out some of the important details about your review as you can use this exercise to document what you have in mind. For more information please see Workbook Instructions in the Introduction Module.]]>

Key Takeaways

fkovacaj — Sat, 15 Jan 2022 17:01:42 +0000

Your research question can be qualitative or quantitative.

A formula like PICO(T) can assist you in creating your question and identifying your key search terms.

Your search terms can include related words and the controlled vocabulary of the subject database.

Using Boolean operators will help you put your search terms together in a coherent search strategy.

Subject databases have limiters that will help you filter your search results.

Use search techniques like citation searching to make sure you have found relevant sources.

If you have a working search strategy, have searched all your identified resources and used citation searching techniques, you can stop searching and start analyzing.

You need to document your search process.

In some cases, more in-depth assistance is required and you can book a research consultation with your subject librarian to review your initial search.

]]>

How to Search

Lefebvre C, Glanville J, Briscoe S, Littlewood A, Marshall C, Metzendorf M-I, Noel-Storr A, Rader T, Shokraneh F, Thomas J, Wieland LS. (2021, February). Chapter 4: Searching for and selecting studies. In Higgins J.P.T, Thomas J, Chandler J, Cumpston M, Li T, Page M.J, Welch V.A. (Eds). Cochrane Handbook for Systematic Reviews of Interventions version 6.2. Cochrane.

Sayers, A. (2008). Tips and tricks in performing a systematic review. British Journal of General Practice, 58(547), 136-136.

Controlled Vocabularies

The National Library of Medicine (2013, February 14). Use MeSH to Build a Better PubMed Query [Video]. YouTube.

UTS Library (2021, February 23). Medline Ovid: Advanced Searching [Video]. YouTube.

CADTH Search Filters Database

Documenting Your Search With PRISMA-S (Systematic Reviews)

Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, et al. PRISMA-S: An extension to the PRISMA Statement for reporting literature searches in systematic reviews. Systematic Reviews. 2021;10(1):39.

The Role of Librarians

Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. Available from Synthesis Manual. Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015). Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. Journal of Clinical Epidemiology, 68(6), 617–626. Spencer, A. J., & Eldredge, J. D. (2018). Roles for librarians in systematic reviews: a scoping review. Journal of the Medical Library Association : JMLA, 106(1), 46–56.]]>

Introduction to Module 3

fkovacaj — Sun, 23 Jan 2022 17:22:28 +0000

rom Flicker user Archivists Round Table. Licensed under Creative Commons CC BY-ND 2.0[/caption] After you have conducted your search, you will be confronted with the dilemma of what to do with your results. It is not unheard of for searches to produce hundreds if not thousands of results to shift through. Your next steps will be to download your results, save them somewhere you can manage them and finally [pb_glossary id="175"]screen[/pb_glossary] them to pick the ones you will be keeping in your review. At this point you should work out a plan for how you will organize your results. Staying organized and managing your results will require a new set of organizational skills, plus these skills will help protect you from accidental [pb_glossary id="176"]plagiarism[/pb_glossary] or [pb_glossary id="160"]bias[/pb_glossary].

Learning Outcomes

By the end of this module, you will be able to:

Export your search results.
Develop a plan to pick a citation management or systematic review tool.
Screen your search results and pick relevant sources for your review.
Recognize how to avoid plagiarism and demonstrate the rules of your chosen citation style.
Consider the use of emerging technologies in the review process.

]]>

Exporting and Saving Results

fkovacaj — Sun, 23 Jan 2022 17:51:46 +0000

permanently store the important bibliographic information or ‘metadata’ attached to your articles - this could include the author name(s), article title, journal title, etc., the article abstract, and any associated keywords.

Exporting

The majority of the databases that you will likely be using for your review will have options to export your references. Some important tips when exporting large amounts of records:

Export your records as an .ris file type, which will allow you to then upload these records to citation managers or systematic review software.
Look for options to include the article abstracts when exporting, as this will be important for the first level of screening.
For systematic reviews: When selecting your records for export, look for options to Select All - depending on the database, you may have to export your references in batches, if there are limits on the number that you can export at a time.

Example

The following video is an example of how to save and export results in EBSCO CINAHL. [embed]https://www.youtube.com/embed/WTixDh4OFcs[/embed] Exporting from EBSCO CINAHL. University of East Anglia Library. Licensed under a Creative Commons CC BY 4.0. *Note this video does not have closed captions, a transcript has been provided below.

[h5p id="18"]

Learning Activity

Complete the following steps to practice exporting search results:

Go to your library’s website and open a subject database you have used in the past, alternatively you can use PubMed (which does not require library access).
Perform a quick search on a topic of your choice.
Familiarize yourself with the exporting options in your database, look for the export button.
Take note of the export styles available.

Create an Account to Save Results

Many database interfaces like EBSCO, OVID and ProQuest allow users to create a personal account. With an account you can save your searches including any keywords, MeSH terms or limiters you used. This means you can return to your account at any time to re-run your search or modify your search and run the search again. See Table 3.1 below on how to set up an account for the following major database vendors (always check with your library to see which databases are available to you as a patron).

Table 3.1. Instructions on Creating a Personal Account to Save Your Search
Database Vendor	Instructions on Setting Up an Account
EBSCO	How to Create & Manage my EBSCOhost account
ProQuest	My Research Account
OVID	Creating a OVID account
Web of Science	Register for an an email

Key Takeaways

Get to know the exporting features of your chosen databases and sign up for an account with each database. This will not only save your results and search history, it will save you time if you have to re-do your search.

]]>

Managing Your Resources With Citation Management Tools

fkovacaj — Sun, 23 Jan 2022 18:07:51 +0000

PRISMA Checklist in Module 2.

Caution: Citation management tools are useful but can produce inaccurate citations. You will need to double check the accuracy of the citation generated by the tool. Therefore, knowing the rules of your citation style is equally important.

Example

The following video provides a quick overview of how a reference management tool works. [embed]https://www.youtube.com/embed/nYDAuT8sSco[/embed] Reference Management Software: How it Works and What it Does. University of York Library. Licensed under a Creative Commons CC BY 4.0. Closed Captions

]]>

Citation Management Products

fkovacaj — Sun, 23 Jan 2022 18:26:12 +0000

Picking a Citation Management System When picking a tool, we suggest trying out a few in order to compare their features. You might also want to keep the following questions in mind when trying them out:

Is the tool within my budget?
Does it work well with the databases I use frequently?
Is it able to organize and filter a large number of citations?
Can it produce a [pb_glossary id="174"]bibliography[/pb_glossary] in the primary style used in my discipline?
If I’m working with a research team, does the software allow me to collaborate with my colleagues smoothly?

Example

If you are interested in a more comprehensive list of citation management tools available worldwide, please see the following chart from Wikipedia: Comparison of Reference Management Software.

Simple Solution: Using Excel/Google Sheets/Numbers

If you have a small and manageable number of sources, you can use a familiar spreadsheet program like Excel, Google Sheets or Numbers to manage your citation data and look for duplicates. You can create your own columns for important citation information like Author, Title, Year, Database, etc. and even a column to enter notes. You can enter your sources manually or use the exporting function found in most databases. For example, databases like PubMed and ProQuest allow you to export the bibliographic information attached to your sources as a .CSV file. This file can be opened as a spreadsheet in Excel, Google Sheets and Numbers. See Figure 3.1 below for an example of a spreadsheet used as a citation management tool. [caption id="attachment_94" align="aligncenter" width="1024"] Figure 3.1. Example of using a spreadsheet for citation management. Source: Kelly Dermody, licensed under a Creative Commons CC BY 4.0[/caption]

Free Software

There are a number of free citation management tools available for you to use. The only requirement is that you sign up for an account and download the product to your device. The three most commonly used tools are Mendeley, Zotero and EndNote Basic. If you are having trouble picking between the three, Table 3.2 below offers a quick comparison chart. For a more in depth comparison you can use the following comparison chart from the University of Toronto Libraries to help you decide.

Table 3.2. Quick Comparison of Mendeley, Zotero and EndNote Basic
Product	Mendeley	Zotero	EndNote Basic
Price	Free	Free and open source	Free* web-based system Provides users with 2GB of online storage *Note: this is the free basic version of EndNote 2.0 which is a paid product.
Availability	Desktop and online	Desktop and online	Online only
Word processor	MS Word Plugin	MS Word and Google Doc Plugin	Download “Cite While you Write” for both Windows and Mac.
Features	Web Importer bookmarklet for most browsers Can save and annotate PDFs Shared folders for collaboration	Web Importer using Firefox, Chrome, or Safari plugin Automatically import PDFs with citations, can add notes to PDFs Shared folders for collaboration	Web Importer bookmarklet for most browsers Add EndNote Click to import PDFs Share folders with other EndNote Basic users
Important links	Create account Download Reference Manager for Desktop Download Mendeley Desktop Help guide	Create account for Zotero Download Zotero Zotero Help guide	Create Account for EndNote Basic EndNote Basic Help Guide EndNote Basic How-To Videos

Paid Products

There are some important reasons for considering a paid product for your choice of citation management tool. They can offer technical support if anything goes wrong, larger storage space, collaboration, duplication and other special features that might meet your needs more than a free product. The following table offers a comparison of two popular paid products available on the market. Some libraries have institutional licenses to one or more of these products, which means as long as you are a student you will have access. Check with your library to see if you have access to these products. Table 3.3 below offers a quick comparison chart of two paid products, RefWorks and EndNote.

Table 3.3. Quick Comparison of Refworks and EndNote
Product	RefWorks	EndNote
Company	Product of Ex Libris, a ProQuest company	Product of Clarivate (owns Web of Science)
Availability	Web based	Desktop interface for Windows and Mac
Word Processor	MS Word and Google Docs plugin	“Cite While You Write” download for Windows and Mac
Features	Read and annotate full-text documents with highlights and comments. Collaborate on projects with shared folders.	Shared folders for collaboration. Add EndNote Click to import PDFs. Conduct large-scale literature reviews with analysis tools. Deduplicate content by searching on unique identifiers.
Important Links	RefWorks Account	EndNote Account

Products Specifically for Systematic Reviews

Some citation managers may not be robust enough to handle a large evidence synthesis review or remove duplicate citations. Instead, you might need to use a tool specifically designed for systematic reviews. These tools incorporate automated features that help organize and speed up the review process. [h5p id="19"] The Systematic Review Toolbox is a searchable collection of tools you could potentially use to develop your systematic review. Below is a sample of some free and paid tools designed for systematic reviews. [h5p id="20"] [h5p id="21"]

Learning Activity

Complete the following steps to locate the tools available to you:

See what paid products your library offers. Bookmark the resource guides on using the product.
Take some time to try out 2-3 products before making a decision or a purchase.

Key Takeaways

Taking some time to pick a citation management tool that is right for you is an essential step in conducting a review. The right tool will help you maintain the integrity of your review and save you time.

]]>

Screening

fkovacaj — Sun, 23 Jan 2022 19:56:36 +0000

Screening is the process of identifying suitable sources from your literature search to be “full-text” screened and eventually included in the review. As mentioned in Module 2, your inclusion and exclusion criteria can help inform which limiters to apply to exclude irrelevant results, e.g., publication year or language. If you are conducting a systematic review, it is recommended to screen the results outside of the databases, as there will be duplication of records. A citation software tool or systematic review tool will aid in removing duplicates. Within the citation management or systematic review tool, you can screen the titles and [pb_glossary id="173"]abstracts[/pb_glossary] to determine if articles are relevant, based on inclusion and exclusion criteria that could not be captured in your search strategy. Afterwards, the full-text must be retrieved and screened to determine inclusion.

Learning Activity

Screening Practice: In this activity, you are doing a review on the effectiveness of cognitive behavioural therapy in improving mild to moderate depression in teenagers. Your inclusion criteria includes:

English language publications only
Longitudinal studies
Published in the last 20 years

Instructions: First, screen the abstract to see if it fits your inclusion criteria. * Don't click on the plus symbols yet.* Then, when you are ready, click on the plus symbols to find out if this article fits with your inclusion criteria. [h5p id="32"] Abstract of Kroll, L., Harrington, R., Jayson, D., Fraser, J., & Gowers, S. (1996). Pilot study of continuation cognitive-behavioral therapy for major depression in adolescent psychiatric patients. Journal of the American Academy of Child and Adolescent Psychiatry, 35(9), 1156–1161. This screen grab from PubMed , National Library of Medicine (NLM) is included on the basis of fair dealing.

]]>

Team Screening for Systematic Reviews

fkovacaj — Sun, 23 Jan 2022 20:03:41 +0000

Screening the Title and Abstract For the first screening, each member of the team will independently scan the titles and abstracts of articles, and make a decision whether to include or exclude studies. Each reviewer should be asking themselves, “Is there enough information in the title and abstract to exclude this study at this stage?” To make sure both reviewers are on the same page, they will need to ensure they have a clear and pre-defined inclusion and exclusion criteria.

Tips for screening the title and abstract:

You can be over-inclusive at this stage.
A “maybe” can be a yes at this point.
Do not screen for outcomes in the abstract or title.

Screening the Full-Text

The next step is to read the full-text of the articles selected from the first round. Again, the reviewers will do this independently and use the same inclusion and exclusion criteria. This stage involves a more rigorous and in depth look at the articles. The reason for excluding a study must be recorded for reporting purposes. Reviewers can also look at the outcome of a study and determine if it is significant enough to include.

How to screen in a citation software management tool with a partner:

Export your library after removing the duplicates and share it with your co-reviewer. This will ensure both screeners assess the literature independently.
Alternatively, see if the citation software manager allows for annotation. If there is only one field that allows for annotation, then each reviewer should consider a separate coding system for inclusion and exclusion.

How to screen in a systematic review tool with a partner:

These tools allow the owner of the account to invite others to participate in the screening process. They usually allow anonymous screening.

]]>

Documenting Your Selection Process and Addressing Bias for Systematic Reviews

fkovacaj — Sun, 23 Jan 2022 20:06:01 +0000

Module 2, you will be expected to document your search process and your selection process. The PRISMA flow diagram will help you map out the number of records identified, included and excluded, and the reasons for exclusions.

Key Takeaways

Save a copy of the PRISMA Flow Diagram for your own review.

Addressing Bias in Selecting Sources for Systematic Reviews

Selection bias can happen consciously or unconsciously to a reviewer when selecting sources. Bias can appear in how and why we make decisions on the potential eligibility or ineligibility of a study. This is why using a team approach is one method to reduce bias in selecting sources for systematic reviews. Another is to make sure each reviewer has a clear understanding of the key concepts, or inclusion and exclusion criteria. For example, has the team checked to ensure everyone has the same understanding of the key concepts in their research question and what to include and exclude? This way, everyone goes into selecting with the same understanding.

Learning Activity

Your group is planning on using the following criteria. What bias or misinterpretation issues might be present within each one? [h5p id="22"]

Key Takeaways

Screening your citations and abstracts against your inclusion and exclusion criteria is the first step in identifying suitable studies. If you are conducting a systematic review, it is important to reduce selection bias by having a blind team based approach to screening.

]]>

Avoiding Plagiarism by Citing It Right

fkovacaj — Sun, 23 Jan 2022 20:14:47 +0000

Plagiarism happens when you claim the words, ideas, or data of another person as if it is your own, without appropriate referencing. Plagiarism can range from copying an entire paper to changing only a few words and not citing the original source. It's important to familiarize yourself with what constitutes plagiarism. One way you can do this is to consult your institution’s academic integrity policy so you know how to avoid a charge of plagiarism.

Citation Styles

When it comes time to write your review, you will need to demonstrate that you understand when and how to cite your sources. Keeping them organized will help you with this task. By using a citation management tool, you will be able to access your sources in one place and pull the [pb_glossary id="172"] citation[/pb_glossary] information needed to cite them correctly. Before sitting down to write your review, make sure you familiarize yourself with the rules of your citation style. While it is true that some citation management tools will offer you a plug-in for your word processor that seemingly does all the work, it can make mistakes. Getting to know the rules yourself will save you a lot of time and reduce the risk of plagiarism. Table 3.4 below provides a curated list of citation style resources to help you familiarize yourself with the rules.

Table 3.4. Citation Style Help Resources
Style	Resource
APA	APA Style.org from APA APA Style and Formatting Guide from Purdue University Online Writing Lab (OWL) APA Citation Style Guide from Concordia University Libraries
MLA	MLA Formatting and Style Guide from Purdue University Online Writing Lab (OWL) MLA 9th Ed from CSUDH University Library MLA Citation Style from Concordia University Libraries
Turabian (Chicago)	Chicago Manual of Style 17th Edition from Purdue University Online Writing Lab (OWL) Chicago Basic Style Guide from Student Learning Support (Toronto Metropolitan University)
Vancouver (also known as ICMJE)	Citing and referencing: Vancouver from Monash University Library ICMJE Recommendations Guide (PDF) from Brescia College University Citing & Referencing: Vancouver Style (PDF) from Imperial College
IEEE	IEEE Reference Guide (PDF) from IEEE Author Centre

Learning Activity

How much do you know about plagiarism? Take this quiz to find out. [h5p id="23"] This quiz is adapted from the University of Guelph, Academic Integrity Plagiarism Quiz. Licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License.

Key Takeaways

Familiarize yourself with the academic integrity policies of your institution to better understand how plagiarism is defined and how to avoid it. Commit to learning the rules or conventions of your chosen citation style.

]]>

Key Takeaways

fkovacaj — Sun, 23 Jan 2022 21:24:07 +0000

Screening your citations and abstracts against your inclusion and exclusion criteria is the first step in identifying suitable studies.

If you are conducting a systematic review, it is important to reduce selection bias by having a team based approach to screening.

Famarizlize yourself with the academic integrity policies of your institution to better understand how plagiarism is defined and how to avoid it.

Commit to learning the rules or conventions of your chosen citation style.

]]>

Screening

Bias

Keenan, C. (2018, April 18). Assessing and addressing bias in systematic reviews. Meta-Evidence Blog, Campbell Collaboration, UK & Ireland.

Citation Mapping

Papaioannou, D., Sutton, A., Carroll, C., Booth, A., & Wong, R. (2010). Literature searching for social science systematic reviews: Consideration of a range of search techniques. Health Information and Libraries Journal, 27(2), 114–122. Tay, A. (n.d.). Citation based literature mapping tools—An update—Tools offering premium accounts, the effects of the loss of MAG and use for evidence synthesis? Aaron Tay’s Musings about Librarianship. ]]>

About This Short Course

fkovacaj — Fri, 28 Jan 2022 00:03:56 +0000

Continuing Studies students at work in the classroom. By Flickr user Simon Fraser University - Communications & Marketing Licensed under Creative Commons CC BY-ND 2.0[/caption]

The Purpose of This Course

This short course is on the research component of conducting a literature or systematic review. The purpose is to spark your intellectual curiosity and develop your research skills. Research is an important part of your review and your work will add to the scholarly discourse of your topic. This self-guided course will help you outline your goals for your review and demonstrate how adopting a mind-set of persistence, adaptability, and flexibility will help you achieve them. Throughout the course, we provide examples and activities designed to help you develop your essential research skills. In Module 1, we provide a quick overview of the different types of academic reviews and the steps involved in conducting them. In Module 2, we demonstrate how to formulate your research question and how to search for sources. In Module 3, you will learn how to select and organize your sources. Finally, in Module 4, we outline techniques for reading and assessing the quality of your sources.

What You Need

An estimated 20 minutes per module (depending on how many of the links to resources you explore, you may take more time)
Access to a computer and the Internet
Access to your institution’s library website
Access to your own notebook or MS Word/Google Doc to take notes

How to Navigate the Modules

We have designed this short course to be done at your own pace and at your own point of need. Therefore, you can navigate to the parts you find relevant to your current situation. This version of the course is hosted in Pressbooks (a WordPress-based online platform). Pressbooks is used to host open textbooks and, in this case, a short course. Please see the video below on how to navigate Pressbooks.

[embed]https://www.youtube.com/embed/2oQCIRORcks[/embed]

Iowa State University Digital Press. Navigating Your Course Pressbook. Licenced under Creative Commons CC BY-NC 4.0. (Closed Captioned)

Research Roadmap Workbook

The workbook is designed to help you reflect and document your research process. Throughout this short course, you will have the opportunity to use the workbook to document aspects of your review. You can refer back to this workbook during the research phase of your review.

How to Use the Workbook

The link below will create a personal copy of the workbook. This workbook is a Google Document and by clicking on the link, a copy of the workbook will save to your Google Drive. This is your personal copy of the Workbook for your own records that you can return to whenever you are prompted to do a workbook activity throughout this short course.

Advanced Research Skills: Conducting Literature and Systematic Reviews Workbook

Learning Outcomes

Below are the learning outcomes for the overall course. Each module will include a list of objectives/outcomes specific to that module to guide you through your learning. By the end of this course, you should be able to:

Understand the specific characteristics of your chosen review and describe the steps involved in conducting a successful review.
Determine your research question and design and refine your own search strategy.
Choose appropriate resources to search and recognize other serendipitous methods of information gathering.
Consider the use of emerging technologies in the review process
Develop a plan to organize your sources.
Screen your search results and pick relevant sources for your review.
Recognize how to avoid plagiarism and demonstrate the rules of your chosen citation style.
Evaluate the relevance of a source to your own context.
Identify patterns in the data and synthesize relevant information
Recognize and acknowledge the role of libraries and librarians in the review process

What This Course Does Not Cover

This course concentrates on the research component of conducting a review. We do not cover how to write a review. Also, we will not cover how to extract and analyze data. In our last section, Resources to Keep you Going, we provide resources that you can consult if you need information on these topics. ]]>

Workbook Activity 0: Review Your Research Skills

fkovacaj — Fri, 28 Jan 2022 00:05:58 +0000

Activity 0: “Review Your Research Skills” in your Research Roadmap Workbook. This survey will help you reflect on which research skills you have and what you need to work on. You can return to this document as you continue through your review to reflect on the progress you have made. For more information please see Workbook Instructions in the Introduction Module.]]>

Workbook To Do List: Research Stage

fkovacaj — Fri, 25 Feb 2022 17:41:01 +0000

To Do List: Research Stage to keep track of your timeline and tasks for the research stage of your review. For more information please see Workbook Instructions in the Introduction Module.]]>

Module 1 Quiz

fkovacaj — Fri, 25 Feb 2022 17:55:13 +0000

[h5p id="6"]]]>

Module 2 Quiz

fkovacaj — Fri, 25 Feb 2022 17:55:59 +0000

[h5p id="17"]]]>

Workbook Activity 3.1: Selecting Your Citation Management Tool

fkovacaj — Fri, 25 Feb 2022 17:44:57 +0000

Activity 3.1: “Selecting Your Citation Management Tool” in your Research Roadmap Workbook. Now that you have been presented with an in-depth list of available tools. Take a moment to document at least 2-3 tools that you are interested in testing. For more information please see Workbook Instructions in the Introduction Module.]]>

Workbook Activity 3.2: Inclusion and Exclusion List

fkovacaj — Fri, 25 Feb 2022 17:46:21 +0000

Activity 3.2: “Inclusion and Exclusion List” in your Research Roadmap Workbook. This exercise allows you to document your criteria. Don’t worry if you are still trying to figure out some of the important details about your review as you can use this exercise to document what you have in mind. For more information please see Workbook Instructions in the Introduction Module.]]>

Workbook Activity 3.3: Check for Team Understanding

fkovacaj — Fri, 25 Feb 2022 17:47:46 +0000

Activity 3.3: “Check for Team Understanding” in your Research Roadmap Workbook. Are there any search terms or items on your inclusion/exclusion list that might have multiple interpretations? Write them down with your own interpretation and have your team members do the same activity. Compare your interpretations to check for team understanding. For more information please see Workbook Instructions in the Introduction Module.]]>

Module 3 Quiz

fkovacaj — Fri, 25 Feb 2022 17:57:02 +0000

[h5p id="24"]]]>

Introduction to Module 4

fkovacaj — Thu, 27 Jan 2022 23:01:31 +0000

Sac State. Licensed under Creative Commons CC BY-ND 2.0[/caption] The purpose of this module is to teach you how to manage your reading workload efficiently, and how to identify the key points in the articles you are including in your review.

Learning Outcomes

By the end of this module, you will be able to:

Identify the various components that should be present in a peer reviewed study.
Evaluate the relevance of a study to your own context.
Identify themes that are relevant to your own research questions.
Identify the contribution that particular articles, books, and other scholarly pieces make to disciplinary knowledge.
Recognize that a given scholarly work may not represent the only (or even the majority) perspective on the issue.
Organize your readings with a literature review matrix
Identify patterns in the data and synthesize relevant information

]]>

The Structure of an Academic Article

fkovacaj — Thu, 27 Jan 2022 23:11:28 +0000

Learning to identify each component is a key step in the strategic reading process, and will help you save time as you screen articles for relevance. Check out the interactive example below that describes each section.

Example

Click on the purple question marks to learn more about each component of an academic article. [h5p id="28"] [h5p id="25"] Structure of an Academic Article by Emma Seston. Licensed under Creative Commons CC BY-NC 4.0.

Key Takeaways

Learning to identify each component is a key step in the strategic reading process, and will help you save time as you screen articles for relevance.

]]>

Initial Scan for Relevance

fkovacaj — Thu, 27 Jan 2022 23:19:59 +0000

Note: this step in the process does not require you to read the entire article! You will begin by scanning the articles and follow these steps:

Example

Click on the purple question marks to learn more about each step in the process of scanning an academic article for relevance. [h5p id="29"] [h5p id="26"] Initial Scan for Relevance by Emma Seston. Licenced under Creative Commons CC BY-NC 4.0.

Key Takeaways

For your first read you can scan the outcome/results or conclusion section and make swift judgments about the relevance of an article to your own context.

]]>

Quality Assessment and Appraisal

fkovacaj — Thu, 27 Jan 2022 23:59:54 +0000

thoroughly.

Critical Questions to Ask When Reading

Critical reading encourages the reader to think about how an article or argument is constructed, not just what it says or what the study found. Here are some questions to ask yourself when reading:

What are the central ideas or arguments in the paper?
Are there words, concepts or research methods used that you don’t understand? If so, ensure you follow up with quality resources to ensure comprehension and extend your learning.
What is the context for the argument? Have they made explicit connections to existing literature, and located their work within that context?
Is their literature review clearly outlined and presented in an inclusive manner? Are there any obvious exclusions? If so, are they accounted for?
Who is the author? Who are they writing for? Have they located themselves and their positionality to the topic at hand? Are there any declared (or perceived) conflicts of interest?
What evidence is given to support the conclusions?
Is the evidence credible, that is, does it come from reliable sources?
Is the logic of the argument sound? What are the steps in the argument that leads from the evidence to the conclusion?
Are any of their ideas problematic? Are there other lenses that could be used to frame the argument?

Tools for Critical Appraisal

Using a standardized tool for critical appraisal is a good practice to follow. It demonstrates consistency in evaluation of all articles, and when working on a research team, it ensures that each individual is asking the same questions. There are numerous tools for appraisal to choose from. For the most part, there will be disciplinary norms where certain tools are chosen in order to best represent the specific context. Looking to other reviews in your field to see what tools are employed is a good step. There are various tools such as checklists designed by research organizations that will help you determine if your source is appropriate. Table 4.1 below outlines a sample of free quality assessment tools available for you to use.

Table 4.1. Quality Appraisal Tools
Tool	Organization	Description
CASP Checklists	Critical Appraisal Skills Programme (CASP) from the Public Health Resource Unit, NHS, England	Checklists for assessing the reliability, importance, and applicability of studies. Studies include qualitative studies, systematic reviews, randomized control trials and others.
Critical Appraisal Tools (CEBM)	The Centre for Evidence-Based Medicine (CEBM) from the University of Oxford	A checklist with questions for assessing the reliability, importance, and applicability of studies based on the research question and inclusion criteria.
Risk of Bias 2 (Rob 2) Tool	Cochrane Methods	A manual from the Cochrane Method Network that includes standards for assessing risk of bias in your included studies.
Critical Appraisal Tools (JBI)	Joanna Briggs Institute (JBI)	Checklists assessing the trustworthiness, relevance and results of different types of studies.
Study Quality Assessment Tools	National Heart, Lung and Blood Institute	Quality assessment tools for assessing potential flaws in study methods or implementation of different types of studies.

To find the tool right for your review check out the Quality Assessment and Risk of Bias Tool Repository from Duke University's Medical Center Library & Archives.

Key Takeaways

While selecting your sources, you will need to assess the quality of the research question, the methodology used, the validity of the results and whether the study is applicable to your own review. To help you assess the quality of your chosen sources, consult one of the free checklist tools suggested in Table 4.1.

]]>

Workbook Activity 4.1: Assessing the Quality of Your Sources

fkovacaj — Fri, 25 Feb 2022 17:49:05 +0000

Activity 4.1: “Assessing the Quality of Your Sources” in your Research Roadmap Workbook. Using a tool for critical appraisal is a good practice to follow. It will help you evaluate your sources and if you are working as a research team, it ensures that each individual is asking the same questions. Take a moment to review some of the tools and document which ones you might use in your review. For more information please see Workbook Instructions in the Introduction Module.]]>

Activity: Choose Your Own Reading Adventure

fkovacaj — Thu, 27 Jan 2022 17:44:24 +0000

Quantitative Study Example Use this Quantitative example study to identify the following components (Katajavuori et al., 2021). Instructions: After taking a guess at the instructions on the front of each card, turn the card to reveal the response. How did you do? [h5p id="30"]

Qualitative Study Example

Read this Qualitative example study to identify the following components (Hickey, et al., 2020). Instructions: After taking a guess at the instructions on the front of each card, turn the card to reveal the response. How did you do? [h5p id="31"]

Hickey, J., Powling, H., McKinney, P., Robbins, T., Carrier, N., & Nash, A. (2020). "It’s a change your life kind of program”: A healing focused camping weekend for urban Indigenous families living in Fredericton, New Brunswick. First Peoples Child & Family Review, 15(2), 23-44. Katajavuori, N., Vehkalahti, K., & Asikainen, H. (2021). Promoting university students’ well-being and studying with an acceptance and commitment therapy (ACT)-based intervention. Current Psychology, 1-13.]]>

Key Takeaways

fkovacaj — Fri, 28 Jan 2022 01:27:47 +0000

Learning to identify each component is a key step in the strategic reading process, and will help you save time as you screen articles for relevance.

For your first read you can scan the outcome/results or conclusion section and make swift judgments about the relevance of an article to your own context.

Finally a literature review matrix will help you organize your readings when it comes time to write your review.

]]>

Quality Assessment

Greenhalgh T. (1997). Assessing the methodological quality of published papers. BMJ (Clinical research ed.), 315(7103), 305–308. Greenhalgh, T., & Taylor, R. (1997). Papers that go beyond numbers (qualitative research). BMJ (Clinical research ed.), 315(7110), 740–743.

Literature Review Matrix

Evidence Synthesis Matrix Template, Jane Schmidt, Toronto Metropolitan University (Google Sheets) The Matrix Method for Literature Reviews, Brandeis University, Writing Resources. Literature Review Synthesis Matrix (MS Word), Concordia University Write a Literature Review: Synthesize. Johns Hopkins University, Sheridan Libraries

Helpful Websites

Systematic Review Guide: Critical Appraisal (St. Michael’s Hospital Health Science Library) Systematic Reviews: Quality Assessment (University of North Carolina Health Science Library) ]]>

Conclusion

fkovacaj — Mon, 31 Jan 2022 17:29:14 +0000

Learning Outcomes

Below are the learning outcomes for this course that we shared with you in the Introduction section. Please take some time to review the list and see how many outcomes you achieved. By completing this short course, you should now be able to:

Understand the specific characteristics of your chosen review and describe the steps involved in conducting a successful review.
Determine your research question and design and refine your own search strategy.
Choose appropriate resources to search and recognize other serendipitous methods of information gathering.
Consider the use of emerging technologies in the review process
Develop a plan to organize your sources.
Screen your search results and pick relevant sources for your review.
Recognize how to avoid plagiarism and demonstrate the rules of your chosen citation style.
Evaluate the relevance of a source to your own context.
Identify patterns in the data and synthesize relevant information
Recognize and acknowledge the role of libraries and librarians in the review process

]]>

Literature Review – More Resources to Keep You Going

fkovacaj — Thu, 27 Jan 2022 23:39:34 +0000

How to Conduct a Literature Review Aveyard, H. (2019). Doing a literature review in health and social care: A practical guide (Fourth ed.) Open University Press, McGraw-Hill Education. Jesson, J., Matheson, L., & Lacey, F. M. (2011). Doing your literature review: Traditional and systematic techniques. SAGE. Ridley, D., Dr. (2008). The literature review: A step-by-step guide for students. SAGE.

Websites

Guide to Literature Reviews (McMaster University Library)

Constructing a Research Question

Alvesson, M., & Sandberg, J. (2013). Constructing research questions: Doing interesting research. SAGE. DeCarlo, M (2018) Chapter 8: Creating and refining a research question in Scientific Inquiry in Social Work. Open Social Work Education. Denney, A. S., & Tewksbury, R. (2013). How to Write a Literature Review. Journal of Criminal Justice Education, 24(2), 218–234.

How to Search

Videos

The National Library of Medicine (2013, February 14). Use MeSH to Build a Better PubMed Query [Video]. YouTube. UTS Library (2021, February 23). Medline Ovid: Advanced Searching [Video]. YouTube.

Websites

Searching for Sources (Research Skills Tutorial), Toronto Metropolitan University Library
Library Research Skills Tutorial, McMaster University Library

Reading Strategies

Greenhalgh T. (1997). How to read a paper. Getting your bearings (deciding what the paper is about). BMJ (Clinical research ed.), 315(7102), 243–246. Sweeney, M. “How to Read for Grad School.” Miriam E. Sweeney, 20 June 2012.

Quality Assessment

Checklists and Tools

CASP Checklists Critical Appraisal Skills Programme (CASP) from the Public Health Resource Unit, NHS, England
Critical Appraisal Tools The Centre for Evidence-Based Medicine (CEBM) from the University of Oxford
Risk of Bias 2 (Rob 2) Tool Cochrane Methods
Critical Appraisal Tools Joanna Briggs Institute
Study Quality Assessment Tools National Heart, Lung and Blood Institute

Writing

Allen, J. (2019). The productive graduate student writer: How to manage your time, process, and energy to write your research proposal, thesis, and dissertation, and get published (First ed.). Stylus Publishing, LLC. Denney, A. S., & Tewksbury, R. (2013). How to Write a Literature Review. Journal of Criminal Justice Education, 24(2), 218–234. Feak, C. B., & Swales, J. M. (2009). Telling a research story: Writing a literature review. University of Michigan Press. Holland, K., & Watson, R. (2021). Writing for publication in nursing and healthcare: Getting it right (Second;2; ed.). John Wiley & Sons Ltd. Walliman, N. (2006). Writing a literature review. In Social research methods (pp. 182-185). SAGE Publications, Ltd.

Website

Writing a Literature Review (OWL, Purdue University)

Official Citation Manuals

APA

American Psychological Association (2020). Publication Manual of the American Psychological Association: The official guide to APA style (Seventh ed.). American Psychological Association.

MLA

Modern Language Association of America. (2021). MLA handbook (Ninth ed.). Modern Language Association of America.

Turabian (Chicago)

Turabian, Booth, W. C., Colomb, G. G., & Williams, J. M. (2018). Manual for Writers of Research Papers, Theses, and Dissertations (9th edition.). University of Chicago Press.

IEEE

IEEE (n.d). IEEE Reference Guide (PDF)

Vancouver or ICMJE

International Committee of Medical Journal Editors. (2021, December). Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals (PDF).]]>

Systematic Review – More Resources to Keep You Going

fkovacaj — Wed, 02 Feb 2022 23:30:49 +0000

How to Conduct a Systematic Review Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19-32. Boland, A., Cherry, M. G., & Dickson, R. (2014). Doing a systematic review: A student’s guide. SAGE. Ganann, R., Ciliska, D., & Thomas, H. (2010). Expediting systematic reviews: Methods and implications of rapid reviews. Implementation Science, 5(1), 56-56. Glass, G. (1976). Primary, Secondary, and Meta-Analysis of Research. Educational Researcher, 5(10), 3-8. doi:10.2307/1174772 Gough, D., & Richardson, M. (2018). Systematic reviews. In Advanced research methods for applied psychology (pp. 63-75). Routledge. Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health information & Libraries Journal, 26(2), 91-108. Higgins J.P.T, Thomas J, Chandler J, Cumpston M, Li T, Page M.J, Welch V.A. (Eds). (2021, February). Cochrane Handbook for Systematic Reviews of Interventions version 6.2. Cochrane.

Websites

Systematic Reviews (Toronto Metropolitan University)
Cochrane Database of Systematic Reviews (Cochrane)
Systematic Review Guide (Unity Health Toronto)
Knowledge syntheses: Systematic & Scoping Reviews, and other review types (University of Toronto Libraries, Gerstein Science Information Centre)

Online Courses on Systematic Reviews

The following guide provides a list of online self-directed learning on how to conduct a systematic review Online Courses on Systematic Reviews (St. Michael's Hospital Health Science Library)

Registering Your Systematic Review

Creating Your Protocol

PRISMA extension for Protocols
MECIR (Methodological Expectations for Cochrane Intervention Reviews) Manual
Sample Protocol Template from the Evidence Synthesis Coordinator at the Maritimes Strategy for Patient-Oriented Research (SPOR) Support Unit.

Documenting Your Review with PRISMA

PRISMA 2020

Constructing a Research Question

Alvesson, M., & Sandberg, J. (2013). Constructing research questions: Doing interesting research. SAGE. Boland, A., Cherry, M. G., & Dickson, R. (2017). Chapter 3 in Doing a systematic review: A student’s guide (Second ed.). SAGE. DeCarlo, M (2018) Chapter 8: Creating and refining a research question in Scientific Inquiry in Social Work. Open Social Work Education. Denney, A. S., & Tewksbury, R. (2013). How to Write a Literature Review. Journal of Criminal Justice Education, 24(2), 218–234. Thomas J, Kneale D, McKenzie JE, Brennan SE, Bhaumik S. (2021, February). Chapter 2: Determining the scope of the review and the questions it will address. In Higgins J.P.T, Thomas J, Chandler J, Cumpston M, Li T, Page M.J, Welch V.A. (Eds). Cochrane Handbook for Systematic Reviews of Interventions version 6.2. Cochrane.

How to Search

Videos

The National Library of Medicine (2013, February 14). Use MeSH to Build a Better PubMed Query [Video]. YouTube.
UTS Library (2021, February 23). Medline Ovid: Advanced Searching [Video]. YouTube.

Websites

Searching for Sources (Research Skills Tutorial), Toronto Metropolitan University Library
Library Research Skills Tutorial, McMaster University Library

Screening

Reading Strategies

Greenhalgh T. (1997). How to read a paper. Getting your bearings (deciding what the paper is about). BMJ (Clinical research ed.), 315(7102), 243–246. Sweeney, M. “How to Read for Grad School.” Miriam E. Sweeney, 20 June 2012.

Bias and Quality Assessment

Boutron I, Page MJ, Higgins JPT, Altman DG, Lundh A, Hróbjartsson A. (2021, February). Chapter 7: Considering bias and conflicts of interest among the included studies. In Higgins J.P.T, Thomas J, Chandler J, Cumpston M, Li T, Page M.J, Welch V.A. (Eds). Cochrane Handbook for Systematic Reviews of Interventions version 6.2. Cochrane. Greenhalgh T. (1997). Assessing the methodological quality of published papers. BMJ (Clinical research ed.), 315(7103), 305–308. Greenhalgh, T., & Taylor, R. (1997). Papers that go beyond numbers (qualitative research). BMJ (Clinical research ed.), 315(7110), 740–743. Keenan, C. (2018, April 18). Assessing and addressing bias in systematic reviews. Meta-Evidence Blog, Campbell Collaboration, UK & Ireland.

Checklists and Tools for Quality Assessment

CASP Checklists Critical Appraisal Skills Programme (CASP) from the Public Health Resource Unit, NHS, England
Critical Appraisal Tools The Centre for Evidence-Based Medicine (CEBM) from the University of Oxford
Risk of Bias 2 (Rob 2) Tool Cochrane Methods
Critical Appraisal Tools Joanna Briggs Institute
Study Quality Assessment Tools National Heart, Lung and Blood Institute

Websites for Quality Assessment

Systematic Review Guide: Critical Appraisal (St. Michael’s Hospital Health Science Library)
Systematic Reviews: Quality Assessment (University of North Carolina Health Science Library)

Tools for Assessing Bias

ROBIS - A tool for assessing the risk of bias in systematic reviews
AMSTAR2 - AMSTAR stands for A MeaSurement Tool to Assess Systematic Reviews.

Analyzing Data

Card, N. A., & Little, T. D. (2012). Applied meta-analysis for social science research. Guilford Press. Cheung, M. W. -., & Vijayakumar, R. (2016). A guide to conducting a meta-analysis. Neuropsychology Review, 26(2), 121-128. Li T, Higgins JPT, Deeks JJ (editors). Chapter 5: Collecting data. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane, 2021.

Writing

Allen, J. (2019). The productive graduate student writer: How to manage your time, process, and energy to write your research proposal, thesis, and dissertation, and get published (First ed.). Stylus Publishing, LLC. Holland, K., & Watson, R. (2021). Writing for publication in nursing and healthcare: Getting it right (Second ed.). John Wiley & Sons Ltd. Boland, A., Cherry, M. G., & Dickson, R. (2014). Doing a systematic review: A student’s guide. SAGE.

Official Citation Manuels

APA

American Psychological Association (2020). Publication Manual of the American Psychological Association: The official guide to APA style (Seventh ed.). American Psychological Association.

MLA

Modern Language Association of America. (2021). MLA handbook (Ninth ed.). Modern Language Association of America.

Turabian (Chicago)

Turabian, Booth, W. C., Colomb, G. G., & Williams, J. M. (2018). Manual for Writers of Research Papers, Theses, and Dissertations (9th edition.). University of Chicago Press.

IEEE

IEEE (n.d). IEEE Reference Guide (PDF).

Vancouver or ICMJE

International Committee of Medical Journal Editors. (2021, December). Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals (PDF).]]>

About the 2nd Edition

fkovacaj — Wed, 01 Mar 2023 05:37:22 +0000

This Second Edition was made possible with funding by the Government of Ontario and through eCampusOntario’s support of the Virtual Learning Strategy. To learn more about the Virtual Learning Strategy visit: https://vls.ecampusontario.ca. ]]>

Accessibility Statement

fkovacaj — Thu, 03 Feb 2022 00:30:33 +0000

Sa11y is an accessibility quality assurance tool that visually highlights common accessibility and usability issues.

Accessibility features of the web version of this resource

The web version of Advanced Research Skills: Conducting Literature and Systematic Reviews includes the following features:

It as been optimized for people who use screen-reader technology:
- All content can be navigated using a keyboard,
- Links, headings, and tables use proper markup, and
- All images have text descriptions.
- Infographics and images with large amounts of text have long descriptions provided below the figure.

Other formats available

In addition to the web version, this book is available in a number of file formats including Digital PDF and EPUB (for eReaders).

Download this book in another file format (Look for the Download this book drop-down menu to select the file type you want.)

Known accessibility issues and areas for improvement

There are no known accessibility issues at this time.

Let us know if you are having problems accessing this book

If accessibility issues are stopping you from accessing the information in this short course, please contact us to let us know and we will get it fixed. If you discover any other issues, please let us know of those as well. Please include the following information:

The location of the problem by providing a web address or page description
A description of the problem
The computer, software, browser, and any assistive technology you are using that can help us diagnose and solve your issue e.g., Windows 10, Google Chrome (Version 65.0.3325.181), NVDA screen reader

You can contact us at kdermody@torontomu.ca This statement was last updated on September 10, 2024]]>

Organize your Readings with a Literature Review Matrix

fkovacaj — Tue, 14 Feb 2023 20:22:51 +0000

Review Matrix One way to organize your literature is with a review matrix. The review matrix is a chart that sorts and categorizes the different arguments presented per topic or issue. Using a matrix enables you to quickly compare and contrast your sources in order to determine the scope of research across time. This will allow you to spot similarities and differences between sources. It is particularly useful in the synthesis and analysis stages of a review (See Module 1 Conducting a Literature Review with the SALSA Framework).

Example of a Review Matrix

My research question: How can we use machine learning to analyze social media data related to HIV?

Sources	Methods	Concept 1	Concept 2	Concept 3	Gaps, Problems, Questions, Notes
Source 1: Signorini, A., Segre, A. M., & Polgreen, P. M. (2011). The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic. PloS one, 6(5), e19467.	Collected and stored a large sample of public tweets that matched a set of pre-specified search terms and geocoded. Estimated rate of disease and public sentiment toward swine flu	Able to make predictions about swine flu using social media data. This data is vital given that “an influenza surveillance program does not exist” (p. 3)			“When and where tweets are less frequent (or where only a subset of tweets contain geographic information), the performance of our model may suffer.”
Source 2: Chiu, C. J., Menacho, L., Fisher, C., & Young, S. D. (2015). Ethics issues in social media–based HIV prevention in low-and middle-income countries. Cambridge Quarterly of Healthcare Ethics, 24(3), 303-310.	Quantitative survey assessing participants’ perspectives on educational intervention	Increasing social media use in low- and middle-income countries. Participant took part in an HIV education program on Facebook	Most participants felt like they benefited positively from the program and learned about HIV prevention. All participants were MSM		Note: Helpful article for including diverse perspectives
Source 3: Bollen, J., Mao, H., & Zeng, X. (2011). Twitter mood predicts the stock market. Journal of computational science, 2(1), 1-8	Collected public tweets and analyzed mood	Gathered data from Twitter posts that explicitly states moods (e.g. “I’m feeling…”). Found that positive/negative sentiment on Twitter is 87.6% accurate for predicting stock market average		Used a “Self-Organizing Fuzzy Neural Network” to predict Dow Jones Industrial Average (p. 1)

Sources

Methods

Concept 1

Concept 2

Concept 3

Gaps, Problems, Questions, Notes

Source 1: Signorini, A., Segre, A. M., & Polgreen, P. M. (2011). The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic. PloS one, 6(5), e19467.

Collected and stored a large sample of public tweets that matched a set of pre-specified search terms and geocoded. Estimated rate of disease and public sentiment toward swine flu

Able to make predictions about swine flu using social media data. This data is vital given that “an influenza surveillance program does not exist” (p. 3)

“When and where tweets are less frequent (or where only a subset of tweets contain geographic information), the performance of our model may suffer.”

Source 2: Chiu, C. J., Menacho, L., Fisher, C., & Young, S. D. (2015). Ethics issues in social media–based HIV prevention in low-and middle-income countries. Cambridge Quarterly of Healthcare Ethics, 24(3), 303-310.

Quantitative survey assessing participants’ perspectives on educational intervention

Increasing social media use in low- and middle-income countries. Participant took part in an HIV education program on Facebook

Most participants felt like they benefited positively from the program and learned about HIV prevention. All participants were MSM

Note: Helpful article for including diverse perspectives

Source 3: Bollen, J., Mao, H., & Zeng, X. (2011). Twitter mood predicts the stock market. Journal of computational science, 2(1), 1-8

Collected public tweets and analyzed mood

Gathered data from Twitter posts that explicitly states moods (e.g. “I’m feeling…”). Found that positive/negative sentiment on Twitter is 87.6% accurate for predicting stock market average

Used a “Self-Organizing Fuzzy Neural Network” to predict Dow Jones Industrial Average (p. 1)

Writing a Literature Review Modified from The WI+RE Team, UCLA. Creative Commons CC-BY-NA-SA

Create a Review Matrix

Start with a charting tool you are most familiar with (for example MS Word, MS Excel, Google Sheets, Numbers etc).

Input your information.
- Organize your sources from oldest to most recent. This way you can see how the research on your topic has changed over time.
Create your columns. The number of columns is up to you, but you will need the following columns to start:
- First Column: citation (i.e., author, title, source, publication year)
- Second Column: purpose or summary (1-2 sentences)
Depending on your review, you can now choose the remaining columns that will help you organize your sources. The columns should represent the specific content you are analyzing. For example:
- methodology
- population
- geography
- intervention
- outcomes

Key Takeaways

Here are some examples of different review matrices and templates:

- Evidence Synthesis Matrix Template, Jane Schmidt, Toronto Metropolitan University (Google Sheets)
- The Matrix Method for Literature Reviews, Brandeis University, Writing Resources.
- Literature Review Synthesis Matrix (MS Word), Concordia University
- Write a Literature Review: Synthesize. Johns Hopkins University, Sheridan Libraries

]]>

Module 4 Quiz

fkovacaj — Fri, 25 Feb 2022 17:57:45 +0000

[h5p id="27"]]]>

Literature review

fkovacaj — Tue, 05 Oct 2021 17:41:05 +0000

Systematic reviews

fkovacaj — Fri, 08 Oct 2021 22:12:51 +0000

Protocol

fkovacaj — Fri, 22 Oct 2021 15:37:21 +0000

Cochrane

fkovacaj — Tue, 26 Oct 2021 00:08:17 +0000

Qualitative

fkovacaj — Tue, 26 Oct 2021 00:11:27 +0000

Synthesize

fkovacaj — Tue, 26 Oct 2021 01:29:40 +0000

Academic Journal / Scholarly Journal

fkovacaj — Tue, 26 Oct 2021 01:33:05 +0000

Peer Review

fkovacaj — Tue, 26 Oct 2021 01:34:16 +0000

literature

fkovacaj — Wed, 01 Dec 2021 01:43:33 +0000

Systematic

fkovacaj — Wed, 01 Dec 2021 01:43:56 +0000

Analyze

fkovacaj — Wed, 01 Dec 2021 02:22:11 +0000

academic journal articles

fkovacaj — Wed, 01 Dec 2021 02:23:36 +0000

About the 3rd Edition

fkovacaj — Tue, 30 Jan 2024 18:18:44 +0000

Advanced Research Skills: Conducting Literature and Systematic Reviews. At the time of completion of the 1st edition, the team of authors could scarcely believe that a 3rd edition would be possible, or necessary. With over 6000 pageviews to date, as well as its use as a course text in several graduate courses, we are thrilled at the reception of this open course. We were also pleased to see it reviewed in the Journal of the Canadian Health Libraries Association Journal in 2022. We have looked closely at the reviewer’s feedback, and have tried to reflect these comments in the 3rd edition, where possible. The comments and feedback of our peers has been invaluable to the revision process, and we thank them all for their interest. We began considering the need for a 3rd edition after reading the comments from our peers, and due to the rapidly growing use of Generative AI (GAI) technology in all facets of academic work, including the production of review papers. We felt it was important to give readers our perspective on the use of GAI in this context, as well as some expert guidance on this ever-evolving technology. We have also included a new section on the use of Citation Mapping Tools in review projects, another type of technology tool that can be used to visualize the research that is connected to a single paper. As with any new or emerging technologies, we always caution care in its use alongside the traditional resources used in review projects. We have also included a new section on Synthesis, always a challenging concept to teach students and researchers. We hope the introduction offered will help readers understand the role synthesis plays in the review process, as well as providing a better understanding of the different approaches that are available. An Evidence Synthesis Chart template is available in this section, which offers readers an experiential learning opportunity to extract the relevant data points from a series of articles, and begin to see the patterns in the data. Finally, as librarians ourselves, The Role of Librarians when supporting review projects is better acknowledged and clarified for our readers in the 3rd edition. Regardless of the type of model of support provided at your library, it is always a best practice to acknowledge the work of librarians providing support to review projects. We hope readers will enjoy the new content available in the 3rd edition, and find it useful as they embark on their own review projects. Kelly Dermody, Cecile Farnum, Daniel Jakubek, Fiona Kovacaj, Jo-Anne Petropoulos, Jane Schmidt and Reece Steinberg]]>

The Role of Librarians

fkovacaj — Wed, 08 May 2024 20:58:26 +0000

Consultant Model or the Co-Authorship Model. In the Consultant Model, the Library provides advice and support to a review project through an in-depth consultation, or a series of consultations. This model provides a time-limited level of support and advice, which could include:

Recommendations on relevant databases and grey literature sources
Advice on developing search strategies
Guidance on the use of citation management tools or systematic review software

Where extensive support has been provided, it is common to ask the librarian how they would like their contributions to be acknowledged. This would typically include a written acknowledgement of their contribution in the published version. See an example: Inglis, S. C., Clark, R. A., McAlister, F. A., Stewart, S., & Cleland, J. G. (2011). Which components of heart failure programmes are effective? A systematic review and meta-analysis of the outcomes of structured telephone support or telemonitoring as the primary component of chronic heart failure management in 8323 patients: Abridged Cochrane Review. European journal of heart failure, 13(9), 1028–1040. In the Co-Authorship Model, a librarian joins your review project team as a co-author. This model provides a more substantial level of support to the project, and could include:

Selection of databases and grey literature resources
Developing the search strategy
Translating the search strategy across all databases
Exporting results to a citation management tool or systematic review software
Writing the methods section specific to searching

See an example: Stotz, S. A., McNealy, K., Begay, R. L., DeSanto, K., Manson, S. M., & Moore, K. R. (2021). Multi-level Diabetes Prevention and Treatment Interventions for Native People in the USA and Canada: a Scoping Review. Current diabetes reports, 21(11), 46.]]>

Introduction to Synthesis

fkovacaj — Wed, 08 May 2024 21:09:32 +0000

One study found that teachers in Ontario believed that play-based learning (PBL) led to increased social inclusion of children with neurodevelopmental delay (NDD).

One study found that a play-based training programme for parents in Hong Kong increased the social skills of their children

One study found that an outdoor play-based learning curriculum in Wales led to increased involvement in tasks and deeper learning when children perceived their lessons as play

One study looked to determine where children would choose to focus their play when given choices of stations; results showed that stations that promoted physical activity were given preference, indicating that when available, children will choose more active play than stationary.

One study found that when play stations encouraged children to write, authentic opportunities for creative expression resulted, but only when intentional teaching took place concurrently

The above list is not a synthesis. It is simply a summary of the findings from distinct studies that have very loose connections. It is now the job of the researcher to try to make connections that may form a cohesive new interpretation of what these other researchers have concluded. This is the process of integration. What are some potential observations that could be made about the above studies?

Play-based learning increased inclusion in the classroom
Participation of adults (both teachers and parents) have measurable effects on outcomes
Play-based learning is effective for learning
Play-based learning is a global practice

None of the example studies drew these conclusions, nor was it the intent of those studies to draw those conclusions. Rather, by analyzing the findings of studies done in very specific circumstances, the intellectual exercise of synthesis allows the researcher to make higher level observations based on the similarities (i.e. they all focus on play based learning) and differences (i.e. they each focus on different participants in different circumstances) in a critical mass of studies that share many similar characteristics, but are not - strictly speaking - specifically about the same thing.

Approaches to Synthesis

There is a great deal of flexibility and individual choice around approaches to synthesis. As stated earlier, this section is not meant to be an exhaustive exploration of the varieties of methods and techniques available; rather, some general guidelines for consideration. Quite broadly, as described by Booth (2012), synthesis relies on pattern recognition in the data. Key considerations would include the comparability of the study settings and the interventions - “in short…whether…it makes sense to treat all included studies as a group (lumping) or whether they make more sense as a series of separate and distinct groups (splitting)” (p. 179). A common approach to synthesis includes narrative approaches which summarize the characteristics and findings of a body of research, primarily using words and text (Booth, 2012). Tabulation can also be used in synthesis; providing a chart of the relevant data points - “eyeballing patterns across rows or down columns.” (p.185). Examples: Thematic analysis is a popular choice, which uses ‘codes’ or ‘themes’ that may be pre-specified or emergent from the data (Gough, 2017). The data is then mapped to these themes and codes in a consistent manner. This approach is common when the synthesis is focussed on qualitative studies. See it in use here: Sartain, S. A., Stressing, S., & Prieto, J. (2015). Patients’ views on the effectiveness of patient-held records: A systematic review and thematic synthesis of qualitative studies. Health Expectations, 18(6), 2666–2677. https://doi.org/10.1111/hex.12240 An alternative approach to a narrative or thematic approach is a meta-analysis, which uses a set of statistical procedures to integrate, summarize or organize a set of reported statistical findings of studies that investigate the same research question using the same methods of measurement.” (Booth, 2012) This methodology is appropriate when there are homogenous amounts of quantitative data under consideration. See it in use here: Ettehad, D., Emdin, C. A., Kiran, A., Anderson, S. G., Callender, T., Emberson, J., Chalmers, J., Rodgers, A., & Rahimi, K. (2016). Blood pressure lowering for prevention of cardiovascular disease and death: a systematic review and meta-analysis. Lancet (London, England), 387(10022), 957–967. https://doi.org/10.1016/S0140-6736(15)01225-8

Evidence Synthesis Chart

Creating an evidence synthesis chart will assist you by compiling and presenting the relevant data points to assist with observing the patterns in the data. Take an article that you’re already familiar with, and practice extracting data into the evidence synthesis chart provided.

Barnett-Page, E., & Thomas, J. (2009). Methods for the synthesis of qualitative research: A critical review. BMC Medical Research Methodology, 9(1), 59. Booth, A., Papaioannou, D., & Sutton, A. (2012). Systematic approaches to a successful literature review. Sage. Gough, D., Oliver, S, & Thomas, J. (2017). An Introduction to Systematic Reviews. Sage. Strike K & Posner G. (1983). Types of synthesis and their criteria. In Ward, S & L. Reed (Eds.), Knowledge structure and use: implications for synthesis and interpretation (pp. 343-362). Temple University Press.]]>

Generative AI & Reviews

fkovacaj — Wed, 08 May 2024 21:22:24 +0000

Issues with Generative AI As with any tool, generative AI is not perfect. There are many issues and limitations, including:

Inaccuracy and errors
Bias and discrimination
Risks to privacy

Inaccuracy and errors

The content that is generated by [pb_glossary id="255"]Large Language Models[/pb_glossary] (LLMs) can be inaccurate or false. LLMs have been known to ‘hallucinate’, giving responses that are false or imaginative (IBM, n.d.). LLMs have also been trained with data up to a certain point in time and cannot account for new information past that cut-off date. For instance, ChatGPT 3.5 does not currently have knowledge of events that occurred after 2021 and can, therefore, produce outdated results (OpenAI, 2024). It’s important to always verify any information that is generated by AI.

Bias and discrimination

LLMs have been created and trained by humans and are therefore not free from bias. Bias can appear in multiple stages, including data collection, data labeling, model training, and deployment (Chapman University, n.d.). LLMs can perpetuate stereotypes and cause discrimination; reinforce exclusion by perpetuating social norms; use harmful language; and can perform better for certain languages and social groups over others (Weidinger et. al., 2021).

Risks to privacy

Training data can include personal and private information which can be revealed by an LLM as a response to a prompt, creating a privacy leak as with Scatterlab’s chatbot Lee-Luda (Weidinger et. al., 2021). As LLMs can also include the information you have provided as part of their training data, that information is also at risk. It’s important to treat all conversations with generative AI models as public, since you cannot control how the information is used once it has been added to a model’s input. Never share sensitive information. These are just three concerns with Generative AI, but there are many others, some of which have been mentioned in the Generative AI Guide.

AI in Literature & Systematic Reviews

The use of Generative AI in academic reviews is a new area of research that is generating interest. Below are some early benefits, cautions and recommendations from van Dijk et al., (2023) unless otherwise stated. A benefit of using an AI tool is that it can save you time compared to other tools and methods. However, we must also consider that:

GAI tools miss some relevant articles for systematic reviews
Deduplication of articles is required, as GAI deduplication is not always accurate

Recommendations:

Be transparent about your use of GAI in reviews: note it prominently in your methodology or equivalent section, and cite the tool(s) you use (See Citations for more information)
Further research and guidelines are needed to ensure quality standards are met using GAI (Cacciamani et al., 2023).

Tools and recommendations are rapidly changing; for current information, please see this guide which includes GAI tools for academic reviews, current research on AI screening of academic reviews and more; updated regularly by authors of this book.

Citations

As mentioned in the section above, transparency is vital if you are using GAI tools in your research, screening and/or writing. Citation is part of this. GAI use in research is so new that style guides may not have current guidance, but many styles such as APA, MLA and Chicago Style have updated recommendations on their websites. For the most current recommendations, please see Citing Artificial Intelligence, updated regularly by authors of this book.

AI Prompts

To interact with a generative AI tool, you will need to give it a [pb_glossary id="254"]prompt[/pb_glossary]. Prompts are written in natural language, which can feel counterintuitive to keyword searching in databases. Typically, the more information you provide, the better. Providing generative AI tools with an effective prompt is an important part of getting the desired response. The following prompt guidance has been adapted from Harvard University Information Technology (2023). Be clear Tell a GAI tool exactly what you would like it to do, as well as what you would like it not to do. Use the words ‘do’ and ‘don’t’ to clarify your criteria. It also helps to be clear about how you would like to receive the output. Maybe you want a list or a couple of paragraphs or in the format of a letter. You can also use examples to provide further clarification, but be careful not to use copyrighted works as an example. Be specific Asking a GAI tool to do something generic like ‘Write a speech’ will produce an equally generic result. To improve this prompt, provide context and background information on your request. You can specify the tone of the response and the audience it is intended for. Typically, the more specific your request the better your result, with the caveat that sometimes being too specific may cause a hallucination as the AI model tries to fill in the blanks of what it does not know. Have a Conversation Part of the appeal of generative AI is that you can speak to it as you would another person and build on your request. If you do not get the results you want or would like the results to be modified, you can provide this feedback to the model. For instance, you could ask a chatbot to: summarize its answer in one paragraph, speak more about a specific aspect of its answer, or write in a more formal language. If you’re stuck on a prompt or would like to improve a prompt, you can also ask the chatbot what it needs from you to fulfill this request. This is part of a process of refining your prompts through trial and error.

Definitions

Deep learning is what makes generative AI and LLMs possible. It “uses neural networks with multiple layers to model and solve complex problems.” (University of Manitoba Libraries, 2024) Generative artificial intelligence is a broad term that encompasses AI systems that generate content. These systems are trained on large amounts of data to produce a response to a user’s prompt. They continually learn and improve on themselves. E.g., Text generators like ChatGPT, image generators like Midjourney, audio generators and video generators like Canva AI. Large Language Model (LLM) is a language model that uses deep learning and large training datasets to recognize, classify, create, predict, and summarize textual content. E.g.. Open AI’s GPT-4 Neural Networks process and analyze data for AI, using algorithms to identify patterns and relationships. They are intended to imitate the operation of a human brain (Chen, 2024). Prompts are the information entered into a GAI tool in order to receive an output. GAI analyzes the prompt, and generates a response based on relationships identified by its neural networks (Harvard University Information Technology, 2023).

Cacciamani, G. E., Chu, T. N., Sanford, D. I., Abreu, A., Duddalwar, V., Oberai, A., Kuo, C.-C. J., Liu, X., Denniston, A. K., Vasey, B., McCulloch, P., Wolff, R. F., Mallett, S., Mongan, J., Kahn, C. E., Jr, Sounderajah, V., Darzi, A., Dahm, P., Moons, K. G. M., … Hung, A. J. (2023). PRISMA AI reporting guidelines for systematic reviews and meta-analyses on AI in healthcare. Nature Medicine, 29(1), 14–15. Chapman University. (n.d.). Bias in AI. Chen, J. (2024, Feb. 7). What Is a Neural Network? Investopedia. Harvard University Information Technology (2023). Getting started with prompts for text-based Generative AI tools. IBM. (n.d.). What are AI Hallucinations? Think. OpenAI. (2024). What is ChatGPT? University of Manitoba Libraries. (2024). Definitions. Using Generative AI for Library Research. van Dijk, S. H. B., Brusse-Keizer, M. G. J., Bucsán, C. C., van der Palen, J., Doggen, C. J. M., & Lenferink, A. (2023). Artificial intelligence in systematic reviews: promising when appropriately used. BMJ Open, 13(7), e072254–e072254. Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., Kenton, Z., Brown, S., Hawkins, W., Stepleton, T., Biles, C., Birhane, A., Haas, J., Rimell, L., Hendricks, L. A., … Gabriel, I. (2021). Ethical and social risks of harm from Language Models (arXiv:2112.04359). arXiv.]]>

Bias

fkovacaj — Wed, 01 Dec 2021 02:38:14 +0000

Quantitative

fkovacaj — Wed, 01 Dec 2021 02:39:57 +0000

and expressed using numbers.]]>

Clinical Research

fkovacaj — Wed, 01 Dec 2021 03:42:20 +0000

Screening

fkovacaj — Wed, 01 Dec 2021 16:04:05 +0000

lived experience

fkovacaj — Wed, 19 Jan 2022 19:45:07 +0000

keywords

fkovacaj — Wed, 19 Jan 2022 20:10:43 +0000

indexing

fkovacaj — Wed, 19 Jan 2022 20:47:27 +0000

Boolean logic

fkovacaj — Wed, 19 Jan 2022 20:55:18 +0000

Truncation

fkovacaj — Wed, 19 Jan 2022 20:58:18 +0000

wildcards

fkovacaj — Wed, 19 Jan 2022 20:59:12 +0000

paywalls

fkovacaj — Wed, 19 Jan 2022 23:05:44 +0000

dissertations

fkovacaj — Wed, 19 Jan 2022 23:07:01 +0000

citation

fkovacaj — Wed, 26 Jan 2022 01:09:26 +0000

abstracts

fkovacaj — Wed, 26 Jan 2022 01:27:53 +0000

bibliography

fkovacaj — Wed, 26 Jan 2022 01:29:12 +0000

screen

fkovacaj — Wed, 26 Jan 2022 01:32:25 +0000

plagiarism

fkovacaj — Wed, 26 Jan 2022 01:33:47 +0000

metadata

fkovacaj — Wed, 08 May 2024 19:33:47 +0000

bibliometric data

fkovacaj — Wed, 08 May 2024 19:46:55 +0000

generative AI

fkovacaj — Wed, 08 May 2024 21:16:58 +0000

Authors