Good topics for systematic reviews

Topic Nomination Development Team

Topic nomination development is typically conducted by a small team consisting of a team lead, research associate, librarian, and clinical team member. The team lead is often a doctorally-trained person with a strong epidemiology, health services research, and systematic review background who provides guidance on the overall content and logic of topic briefs. The research associate is usually a masters level or higher researcher with an epidemiology, biological sciences, or public health background. S/he does the bulk of the work, including the background searching, definition of the topic scope, documentation of the existing guidance, synthesis of the systematic review search and feasibility scan, and evaluation of the topics fit with the EHC Program selection criteria. A masters level research librarian conducts the systematic review searches and feasibility scans.

The team should also include a generalist clinical team member with expertise in systematic reviews. This team member dedicates 15 hours for each topic nomination answering questions from research associates, consulting clinical specialists, and reviewing topic briefs. This team member helps interpret the nomination and clarifies practice variation, clinical uncertainty, appropriate comparators, important subpopulations and outcomes, and other aspects of the topic necessary to understand the current practice or health policy context underlying the need for synthesized research. Generalist physicians can address many questions, supplemented by specialist input for clinical issues not typically handled in primary care. After completion of the topic brief, it is extremely helpful to ask this clinical team member to review the logical flow of evidence that supports the teams recommendation for the topics disposition. Clinical team members can also help identify potential partners for topics.

Topic Nomination Development Process Overview

The topic nomination development process begins with the receipt of a nomination via the EHC Program Web site [//effectivehealthcare.ahrq.gov/index.cfm/submit-a-suggestion-for-research/]. The steps in this process are geared towards evaluating how a topic nomination fits the aforementioned EHC Program selection criteria []. The five main domains of criteria include [1] the appropriateness of a topic, including how it fits within the mandate and priority conditions of the EHC Program []; [2] the importance to the U.S. population and health care system; [3] desirability [or lack of duplication] of a new systematic review; [4] feasibility; and [5] potential impact of a new research review for the topic. These five domains of the criteria are evaluated in order []. If the topic meets none of the appropriateness criteria, the other criteria are not considered. If a topic meets appropriateness and importance criteria, but is duplicative with an existing review or is not feasible for a new review, the potential impact of a new review is not relevant so these criteria are not assessed. Evaluating each nominations fit with these selection criteria using a consistent, transparent process helps ensure that all nominations are treated equitably.

shows the steps in the topic nomination development process; each step is discussed in detail below. The process is not as linear as this diagram implies; many of the steps in the process are integrated and may occur simultaneously. The process includes background searching, definition of the topic scope, a search for systematic reviews, documentation of existing guidance on the topic, a feasibility scan for primary research, and completion of a three part topic brief summarizes information relevant to the topics evaluation against EHC Program selection criteria.3

The topic brief, comprising an Existing Guidance document, Selection Criteria table, and Cover Sheet, allows consistent organization of information to assist orderly, efficient consideration of the topic. The Existing Guidance document lists available and in-process research on the topic. The Selection Criteria document is a table detailing how the topic meets or does not meet each of the EHC Program selection criteria in . The Cover Sheet, typically seven to eight pages long, includes a description of the nomination, background information on the topic, and a summary of the topics fit with EHC Program selection criteria. Thus, the three main documents are relatedthe existing guidance on a topic helps determine the topics fit with the selection criteria, and the topics fit with the selection criteria is summarized in the Cover Sheet. Supplementary materials, such as summary tables of existing systematic reviews and/or clinical guidelines on the topic, may be included as appendices in the topic brief.

also shows the steps in the topic selection process. Selection of topics for systematic review occurs at monthly meetings of a topic triage group representing stakeholder and scientific perspectives, as well as the programmatic authority vested in AHRQ, and is informed by the information presented in the topic briefs. Results of the topic selection process are described in a Nomination Summary Document [described below] to communicate the disposition of nominations to the public.

Prior to nominations beginning topic nomination development, the Scientific Resource Center and AHRQ conduct an initial assessment of nominations to determine whether they meet EHC Program appropriateness criteria and contain enough information to proceed with topic nomination development. Some nominations to the EHC Program lack sufficient information to evaluate the topic against EHC Program selection criteria so do not undergo topic nomination development. Many of these nominations consist of only a few words or are extremely broad, with no indication of how the nomination could be translated into a feasible topic with well-defined populations, interventions, comparators, and outcomes [PICO]. In such cases, we refer to an established checklist for the minimum amount of information needed to evaluate a nomination. This set of minimum information includes the populations, interventions, comparators, and outcomes of interest to the nominator, as well as the policy and/or clinical context. It is sometimes possible to logically conclude what these parameters are, based on the literature and consultation with clinical members of the topic nomination development team, so the nomination can go through the topic nomination development process. In other cases, further input from the nominator is necessary, but is not always possible if the nomination was made anonymously or the nominator does not respond to requests for clarification. In addition, the scope of some nominations may be too broad to develop given limited EHC Program resources.

Background Searching

After reviewing the information provided in the topic nomination, topic nomination development begins with a brief background scan to get a firm understanding of the context, clinical practice and health implications of the topic, what tests or treatments are available, the terms and language used to describe the topic, and affected individuals or populations [with attention to health disparities]. Suggested sources for this search include PubMed for recent narrative reviews, clinical library sources [e.g., DynaMed, First Consult, MD Consult, BMJ Clinical Evidence], U.S. FDA Web site, Centers for Disease Control [CDC] and National Center for Health Statistics fast stats for epidemiology and health statistics, National Cancer Institute Physician Data Query, and relevant professional society Web sites. This background information informs the nominations fit with the appropriateness and importance criteria. This information also aids in the determination of whether the scope of the topic as described in the nomination is clinically appropriate and relevant, and informs development of the preliminary PICO for the topic, which guides the remainder of the topics development.

Definition of the PICO/Scope

In addition to variation in clinical context and nominator perspective, nominations differ greatly in their scope [i.e., the parameters of the research question to be included as defined by the PICO]. A poorly defined PICO can lead to development of a topic that may miss important populations, lack clinical relevance or logic, or is not feasible for a systematic review. During topic nomination development, a number of different sources can be used to further define a topics PICO, including published literature, clarification from the nominator, and consultation with clinical experts; these sources are used to ensure that the PICO is clinically logical and relevant, includes a realistic set of parameters for a potential review, and would result in a review that is useful to important stakeholders. For the purposes of topic nomination development, the formulation of a PICO is done routinely; timing and setting[s] [PICOTS] may be included if these details are important to the context of the nomination. The literature usually suggests the relevant parameters for a topic, which are compared with the PICO proposed by the nominator. Substantial differences can be reconciled during discussions with the nominator to ensure that the nominators interests are reflected in the PICO, and consultation with clinical experts can serve to confirm or revise the PICO to be certain that it fits with the current clinical context.

A vague PICO also presents scoping challenges and may lead to a review that is too inclusive or too exclusive. A narrow PICO may reflect proprietary or individual interests that are not broadly generalizable. A broad PICO is often too imprecise for careful consideration, masking important questions or topics for systematic review. There is an inherent tension in the scoping process between fidelity to the original nomination and broadening the scope of the topic to be more relevant to a larger audience. Discussions with the nominator and other important stakeholders serve to ensure that the nominators interests are clearly articulated in the topic brief along with the evidence needs of other key stakeholders for the topic, such as clinicians or policymakers. For example, a nomination on physical therapy for acute ambulatory conditions was too broad to develop or evaluate against EHC Program selection criteria because the interventions, assessments, and outcomes are heavily dependent upon the specific condition for which there is an indication for physical therapy. The physical therapy literature helped identify the most common conditions for which physical therapy is used. Conversations with the nominator facilitated by a clinical expert in the field of physical therapy clarified that the condition of most interest to the nominator was knee pain secondary to osteoarthritis. Further consultation with the nominator narrowed the nominators questions to focus on issues such as the relationship between intermediate outcomes and improvement in patient functional performance. On the other hand, a nomination on the effectiveness of a combination of IV diphenhydramine, ketorolac, and metoclopramide in addition to saline intravenous fluids for treating acute migraines in emergency settings was too narrow based on the lack of literature on this drug combination and clinical input. The topic was expanded to more broadly address interventions for the treatment of acute migraines, thus scoping the topic in a manner suited to a review that would be useful for multiple stakeholders, including patients, clinicians, policymakers, and guideline developers. Another common scoping problem is that children and other relevant subpopulations may be omitted in the nomination.

Determination of the clinical context or clinical logic has also been a challenge. For example, in a nomination on benign prostatic hyperplasia, the nominator was mainly concerned with the use of complementary and alternative medicine for benign prostatic hyperplasia for the reduction of prostate-specific antigen levels. However, prostate-specific antigen levels are not a clinically relevant outcome for the topic. Our workup was revised to reflect relevant outcomes we found in the literature and confirmed by clinical consultation.

During consultations with clinical experts, it is useful to ask questions such as where the nominators intervention of interest falls within the usual management of the given condition; what other interventions are potential comparators; what outcomes are clinically meaningful for a given intervention or comparator; and whether the intervention of interest is currently used in clinical practice and, if so, how often and in what patient populations it is used most widely.

To aid topic selection decisions, a well-defined PICO should include the following

• Details on the population [e.g., age, sex, disease stage/severity, subpopulations of interest]

• Comprehensive list of interventions and comparators when the nominator has only provided a general category or class

• Definition of usual standard of care if used as a comparator

• List of intermediate and health outcomes, including potential benefits and harms of interventions and comparators, with particular attention to patient-oriented, clinically relevant, and long-term outcomes

presents examples of a poorly-defined and a well-defined PICO.

Search for Systematic Reviews

Searching for literature to answer the nominators question is usually conducted in a sequential manner, beginning with synthesized literature identified from a formal search of medical literature databases, then research products and activities identified from searches of specific organization and agency Web sites described below under Existing Guidance Documentation, and later moving to formal searches for trials and other study designs as described below under Feasibility Scan.

Searching begins with identification of existing and in-process systematic reviews and meta-analyses. This search is conducted by a librarian, but it is helpful to provide the librarian with a list of suggested search terms, including Medical Subject Headings [MeSH] and key words, based on the initial background scan, as well as the databases to search [e.g., MEDLINE, PsycINFO, Cumulative Index to Nursing and Allied Health Literature [CINAHL]] and citations that are good illustrations of the topic [e.g., high-quality narrative or systematic review identified in initial background scan]. The search strategies for existing systematic reviews are a good starting point for development of this search. Based on prior experience, a search for synthesized literature over the past 5 years is often sufficient, although a search of the past 10 years is necessary for some topics, such as those related to well-established interventions that have not been the focus of recent research activity or topics with limited existing research. In some cases, search dates are dependent upon when the technology or intervention was first developed.

After receiving synthesized literature search results from the librarian and reviewing relevant abstracts, an iterative process begins to determine if the search for synthesized research is adequate and captured the questions raised in the nomination or needs to be narrowed or refined. Citations for the most recent, relevant systematic reviews should be listed in the Existing Guidance document, including the search dates, methods, and overall fit with the nomination.

The ultimate goal of this step is evaluation of the duplication selection criterion. In order that EHC Program resources are put to the best use, the EHC Program may decide not to pursue systematic reviews on topics that are already addressed by existing or in-process high-quality reviews. Such a decision does not constitute endorsement of non-AHRQ systematic reviews, but rather the recognition that there are many important topics in health care that would benefit from systematic evidence reviews and only limited resources with which to do those reviews. The EHC Program may consider a topic as adequately covered by a recent review performed or commissioned by a U.S. government agency [e.g., AHRQ, U.S. Preventive Services Task Force [USPSTF], National Institutes of Health [NIH], Department of Veterans Affairs [VA], CDC] or an independent center, academic institution, or government [e.g., Cochrane Collaboration, National Institute for Health and Clinical Excellence [NICE], Canadian Agency for Drugs and Technologies in Health [CADTH], other center or independent group] using acceptable methodology for evidence grading and conflict of interest management. In some cases, the EHC Program may decide to undertake a review despite possible duplication for reasons such as

• A U.S. government product is needed for development of guidelines, policy, or translational products for patients or clinicians.

• Impact will be ensured by use of the AHRQ dissemination infrastructure.

• There are potential benefits from expanding or revising the methodology or better managing conflict of interest in the existing review.

• The existing review was conducted in another country where practice patterns or epidemiology are significantly different than what would be found in the United States or conclusions are not consistent with U.S. guidelines.

• Current clinical practice diverges from consistent conclusions from recent systematic reviews.

• Existing systematic reviews have conflicting conclusions.

• The nominator confirms that current reviews do not meet stated needs.

Existing Guidance Documentation

This step focuses on searching for available and in-process research [e.g., reviews, guidelines, studies] and activities [e.g., Centers for Medicare & Medicaid Services [CMS] policies, NIH conferences] related to the topic, which is recorded in the Existing Guidance document along with the results of the more formal librarian searches for systematic reviews [described above] and primary studies [described below under Feasibility Scan]. The existing guidance informs the evaluation of the topics fit with some of the EHC Program selection criteria, such as duplication, feasibility, and potential impact. Documentation of existing guidance on the topic typically begins while the librarian is conducting the search for systematic reviews. The sources searched for existing guidance include

• In-process and completed AHRQ products

  • Evidence reviews [from Evidence-based Practice Center and EHC Programs]

  • Developing Evidence to Inform Decisions about Effectiveness [DEcIDE] Network projects

  • Translational products [e.g., patient and clinician guides]

• Cochrane Collaboration reviews and protocols

• Drug Effectiveness Review Project drug class reviews

• Health technology assessments [from Centre for Reviews and Dissemination database, which includes content from the International Network of Agencies for Health Technology Assessment and 20 other health technology assessment organizations]

• PROSPERO database of registered systematic reviews and protocols

• VA products [technology assessments from the VA Technology Assessment Program, systematic reviews from the Evidence Synthesis Program, and VA/Department of Defense Clinical Practice Guidelines]

• NIH consensus statements and upcoming conferences

• CDC Guide to Community Preventive Services publications and recommendations for public health topics

• CMS policies and coverage updates

• ClinicalTrials.gov for active, recently completed, or recruiting studies

• National Guideline Clearinghouse at guidelines.gov and other searches [e.g., PubMed] for clinical practice guidelines

In order to show the breadth of existing or in-process AHRQ activities in the clinical domain, AHRQ products that are related to but dont directly overlap with the nomination should be documented. For example, there may not be any AHRQ products addressing a nomination for complementary and alternative medicine [CAM] therapies for sleep apnea, but all AHRQ products on sleep apnea should be documented. A comprehensive list of related AHRQ products can also serve as a reference of those who have worked on similar topics and could potentially serve as experts during later stages of the topic nomination development, refinement, or review process. It is also helpful to document the Key Questions for all relevant AHRQ reviews to illustrate whether the existing AHRQ reviews appear to address the full scope of the nomination.

Feasibility Scan

After the search for systematic reviews, a search for controlled trials is conducted by the librarian to determine the feasibility of a new review on the topic. The dates for this scan can begin from the last search date of the most recent high-quality systematic review. The results of the feasibility scan will show whether the most recent systematic review fully covers the topic. If there are landmark studies or a significant number of studies that have not been captured in the most recent systematic review, the need for a new review on the subject should be considered. The recent introduction of new interventions or technologies for which there is published evidence may also underscore the need for a review on the topic. In the absence of a recent high-quality systematic review, a feasibility scan of the last 5 or 10 years will be needed to determine the adequacy, type, and volume of primary research recently published on the topic that would be available for a review. For those topics with a very limited literature base, a search may need to be completed without date limits. If very few controlled trials are found or for topics that are not appropriate for controlled trials, the feasibility scan should be expanded to include study designs such as case-control, cohort, before-after, case series, and other observational designs. The sufficiency of available studies to warrant a review will partly depend on the topic. For topics where controlled trials are possible but only observational studies are available, a review may not have significant clinical impact until there is higher quality evidence on the topic. For other topics, such as those focused on potential harms, data from observational studies may be sufficient for a review.

The aim of the feasibility scan is only to provide a sense of the volume of the available literature that could potentially be included in a review. This scan is geared toward efficiency and is not meant to be as rigorous as a search for primary literature that is conducted during the course of a systematic review. For topics that are selected for a systematic review, more precise searches will be conducted during the conduct of the systematic review that reflect scope revisions made during the topic refinement process. Synthesis of the feasibility scan results is limited to a summary of the number of relevant studies available for inclusion in a review and documentation of any landmark studies. Unlike synthesis of the results of searches for primary research conducted during a systematic review, synthesis of feasibility scan findings during topic nomination development does not include quality rating of articles or an assessment of the results of the studies. High volume or very broad feasibility scan results are a challenge for some nominations. These cases require organization of the results by the most important parameters of the particular topic, such as setting, population, outcomes, comparators, study design, or length of followup, to aid in the determination of whether the existing literature covers all aspects of the nomination. For example, the feasibility scan results for a topic on fibromyalgia treatment were categorized by the type of intervention studied, including pharmacological, psychological, exercise, and CAM therapies, and for a topic on seasonal allergy treatments they were divided by studies addressing adults versus children.

Completion of Topic Brief

Stakeholder Engagement

shows the points of stakeholder engagement during topic nomination development. In this context, stakeholders are defined as clinicians, policymakers, guideline developers, professional societies, consumers, and patients; the individual nominator may represent one or more of these stakeholder groups. Input from nominators is sometimes needed to clarify the population, interventions, or outcomes of interest when the nomination includes a broad scope or less-defined PICO. In addition, if a topic is deemed duplicative with in-process or existing reviews or programmatic activities, it is sometimes important to verify that the existing products meet the nominators needs. This can occur before the topic is presented to the topic triage group, after presentation to the topic triage group but before final disposition of the topic, or during topic refinement, and EHC Program staff usually determine the appropriate time for this engagement with the nominator. As mentioned above, discussions with local, regional, or national clinical experts are often necessary to appropriately scope a topic at the beginning of topic nomination development, and these discussions occur at the discretion of the topic nomination development team. Experts are generally identified by the clinical team member, who communicates with these experts via email or phone. EHC Program staff may provide guidance to the topic nomination development team as to whether and when the nominator, policymakers, or professional society representatives should be consulted.

Stakeholder input can often be solicited via email, although longer conversations are sometimes required that are better handled on the phone after an initial request for information over email. More formal telephone conferences facilitated by clinical team members are occasionally appropriate to clarify nominations from professional societies or policymakers. For topics voted forward for a systematic review, it can be useful to establish a partnership with a group that will develop clinical practice guidelines based on the review to ensure clinical impact and facilitate dissemination. In such cases, communication with the partnering organization is essential to ensure that the timing of the reviews completion is coordinated with guideline development.

Efficiency

The need for and importance of topic nomination development to identify the most important topics for systematic review is unquestionable. But allowing a longer timeline for in-depth topic nomination development comes at the expense of extending the time between submission of nominations and their disposition. Ultimately, spending more time on topic nomination development may lengthen the timeline for completion of any commissioned reviews and translational products or clinical practice guidelines produced from the reviews. Topic nomination development for the EHC Program is time intensive because it requires a universal perspective given the public funding for products that could be important to several segments of the population. As mentioned above, nominations to the EHC Program cover a broad range of clinical conditions and are submitted by a wide array of stakeholders with varying perspectives and needs, thus, a significant amount of effort is required to find a clear context for each topic. The time needed to complete the steps in the current topic nomination development process varies considerably depending on the complexity and breadth of the topic nomination, with the total time for completion of a topic brief ranging from 16 to 68 hours. This estimate does not include time needed for feedback loops such as going back to the nominator for clarification or getting expert feedback. The EHC Program receives an average of nine nominations per month. Eight nominations on average are triaged per month, and the mean time from nomination submission to triage is 7 months. Balancing efficiency with the need for a comprehensive, effective process will continue to be a challenge and will require exploration of potential process revisions, such as instituting a streamlined process for nominations that are clearly covered by existing programmatic activities [e.g., in-process EHC Program reviews, USPSTF recommendations].

Evaluation of Nominations for New Research

Another challenge encountered in topic nomination development for the EHC Program is presented by nominations for new primary research, which are ill-suited to the existing process for selecting topics for research reviews. In 2011, the Scientific Resource Center and AHRQ adapted the EHC Programs process for evaluation of topics for systematic review to distinguish topics appropriate for potential new research. Potential new research topics are characterized by the existence of a significant research gap that is important to clinician, policymaker, and/or patient decisionmaking. In this process, research gaps and the potential impact of new research on clinical practice and policy are identified by examining the following:

• Systematic reviews5 and editorials for any discussion of research gaps

• Clinical practice guidelines for areas reported as having insufficient evidence to make a recommendation

• Recently published studies to determine to what extent research gaps have been filled

• In-process studies and newly funded Federal research or funding opportunities to get a sense of whether it is an active area of research

• Coverage determinations that provide a perspective on uncertainty surrounding a topic.

Clinical consultation is used to confirm a lack of evidence and the need to rely solely on clinical judgment. This background information on the need for new research on a topic is included in the Nomination Summary Document that is sent to the nominator and posted on the EHC Program Web site [see below]. Evidence generation programs at AHRQ, such as the DEcIDE Network, as well as researchers, funders, and programs outside of AHRQ, can access this information to support their primary research agendas.

Video liên quan