Towards a Framework for Communicating Confidence in Methodological Recommendations for Systematic Reviews and Meta-Analyses – AHRQ – September 2013

Posted on October 23, 2013. Filed under: Research | Tags: , , |

Towards a Framework for Communicating Confidence in Methodological Recommendations for Systematic Reviews and Meta-Analyses – AHRQ – September 2013

Trikalinos TA, Dahabreh IJ, Wallace BC, Schmid CH, Lau J. Towards a Framework for Communicating Confidence in Methodological Recommendations for Systematic Reviews and Meta-Analyses. Methods Research Report. (Prepared by Tufts Evidence-based Practice Center under Contract No. 290-2007-10055-I). AHRQ Publication No. 13-EHC119-EF. Rockville, MD: Agency for Healthcare Research and Quality; September 2013

“Abstract

We propose a framework for organizing and describing the rationale behind methodological recommendations, and for communicating one’s confidence in them. We start by defining the background context in which the recommendations are made. We distinguish recommendations that are testable (in that their likelihood to hold can be informed by theoretical arguments or empirical data) from nontestable ones, which represent beliefs or assumptions that are not verifiable. Nontestable statements can be justified, but their validity cannot be demonstrated. Testable statements can be assessed in terms of the adequacy of their evidentiary basis.

Both testable and nontestable statements can be evaluated regarding their feasibility of implementation, the expected impact of following them versus not, and their congruence with the desired characteristics of the background context. Considering these four dimensions, one can indicate one’s confidence (along a continuum) about how closely a methods recommendation should be followed: some recommendations may be perceived and communicated as mandatory items (minimum standards), while others as highly desirable but not mandatory items. Finally, giving specific methods guidance for addressing difficult or ill-defined problems can be premature pending more research or clearer problem specification. In such cases, describing the problem and laying out attributes of a satisfactory resolution can serve until actionable guidance can be offered.

We view the proposed framework strictly as a communication tool to describe rationale for the recommendations to the intended audience and not as a device to deduce the “correctness” of a recommendation. Nonetheless, application of the framework can facilitate the latter, because methodologists can most effectively and honestly critique recommendations whose rationale is transparent. We anticipate that this initial instantiation of the framework for making methods recommendations will evolve.”

Advertisements
Read Full Post | Make a Comment ( Comments Off on Towards a Framework for Communicating Confidence in Methodological Recommendations for Systematic Reviews and Meta-Analyses – AHRQ – September 2013 )

Inclusion Criteria [for clinical practice guidelines to be included in the US National Guideline Clearinghouse] – June 2013 Agency for Healthcare Research and Qualit

Posted on June 11, 2013. Filed under: Clin Governance / Risk Mgmt / Quality | Tags: , , , , |

Inclusion Criteria [for clinical practice guidelines to be included in the US National Guideline Clearinghouse] – June 2013  Agency for Healthcare Research and Quality

“Effective June 2014, NGC will employ the 2011 definition of clinical practice guideline developed by the Institute of Medicine (IOM).1

Clinical practice guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.

For more information, please refer to the Frequently Asked Questions. We invite you to send your comments to info@guideline.gov.

Revised Criteria for Inclusion of Clinical Practice Guidelines in NGC

Effective June 2014: In order for NGC to accept a submitted clinical practice guideline, the guideline must meet all the criteria below. In addition to the guideline, developers must provide NGC with documentation of the underlying systematic review*.”

… continues on the site

 

Read Full Post | Make a Comment ( Comments Off on Inclusion Criteria [for clinical practice guidelines to be included in the US National Guideline Clearinghouse] – June 2013 Agency for Healthcare Research and Qualit )

Framework for Determining Research Gaps During Systematic Review: Evaluation – Agency for Healthcare Research and Quality [US] – February 2013

Posted on March 15, 2013. Filed under: Research | Tags: , |

Framework for Determining Research Gaps During Systematic Review: Evaluation – Agency for Healthcare Research and Quality [US] – February 2013

Methods Research Reports

Investigators: Karen A Robinson, PhD, Oluwaseun Akinyede, MPH, Tania Dutta, MS, MPP, Veronica Ivey Sawin, BA, Tianjing Li, MD, PhD, Merianne Rose Spencer, BS, Charles M Turkelson, PhD, and Christine Weston, PhD.

Johns Hopkins University Evidence-based Practice Center
Rockville (MD): Agency for Healthcare Research and Quality (US); February 2013.
Report No.: 13-EHC019-EF

“Structured Abstract

Background:
Research gaps prevent systematic reviewers from making conclusions and, ultimately, limit our ability to make informed health care decisions. While there are well-defined methods for conducting a systematic review, there has been no explicit process for the identification of research gaps from systematic reviews. In a prior project we developed a framework to facilitate the systematic identification and characterization of research gaps from systematic reviews. This framework uses elements of PICOS (Population, Intervention, Comparison, Outcomes, Setting) to describe the gaps and categorizes the reasons for the gaps as (A) insufficient or imprecise information, (B) biased information, (C) inconsistent or unknown consistency results, and/or (D) not the right information.”

continues

Read Full Post | Make a Comment ( Comments Off on Framework for Determining Research Gaps During Systematic Review: Evaluation – Agency for Healthcare Research and Quality [US] – February 2013 )

Methods for Benefit and Harm Assessment in Systematic Reviews – AHRQ – November 2012

Posted on November 20, 2012. Filed under: Evidence Based Practice, Research | Tags: , |

Methods for Benefit and Harm Assessment in Systematic Reviews – AHRQ – November 2012

“Structured Abstract

Introduction. Systematic reviewers are challenged by how to report and synthesize information about benefits and harms of medical interventions so that decisionmakers with varying preferences can better assess the balance of benefit and harm. Quantitative approaches exist for assessing benefits and harms, but it is unclear whether they are applicable to systematic reviews.

Objectives. The objectives of this report are: (1) to describe the challenges of quantitative approaches for assessing benefits and harms, (2) to describe methodological characteristics of existing quantitative approaches for assessing benefits and harms, (3) to determine the role of values and preferences in assessing benefits and harms across each step of a systematic review and (4) to formulate principles for assessing benefits and harms in systematic reviews.

Process. We formed a multidisciplinary team with expertise in clinical medicine, systematic reviews, statistics, and epidemiology. The team reviewed the literature on quantitative approaches for assessing benefits and harms of medical interventions, and held 12 weekly meetings to establish consensus about: 1) the challenges in assessing benefits and harms; 2) the methodological characteristics of approaches that have been used; and 3) the role of values and preferences when assessing benefits and harms in systematic reviews.

The team used that information to formulate principles for analyzing benefits and harms in systematic reviews so that decisionmakers are able to weigh the benefits and harms for a given population. An external panel of experts provided input in this process.

Results. Our team identified numerous challenges for the assessment of benefits and harms. The main challenges relate to selection of health outcomes important to patients, information asymmetry (e.g., reliable and robust data on benefits with sparse data on harms), and calculation of statistical uncertainty if benefit and harm are put on the same scale using a benefit harm comparison metric, and consideration of patient preferences.

We identified 16 quantitative approaches for the assessment of benefits and harms. Twelve of the methods can be used in a systematic review because the methods can be applied with the types of summary data that are typically reported and do not require individual patient data. Simpler approaches, such as the ratio of the number needed to treat to the number needed to harm, may be suitable for relatively simple decisionmaking contexts where relevant benefit and harm outcomes are few in number and similar in importance. More complex approaches are needed for decisionmaking contexts having a large number of relevant benefits and harms.

For individual-level decisions, values and preferences are key for determining the balance of benefit and harm. Choices are made by decisionmakers that are informed by the preferences of patients and other considerations. These choices, and therefore preferences, have an important role in determining how benefits and harms are assessed in systematic reviews. These choices and preferences also affect how guideline developers frame recommendations, how regulatory bodies make decisions at the population level, and how clinicians, patients, and other end users make decisions at the individual level.
The team formulated principles to conduct comparative assessments of benefits and harms in the context of a systematic review. For example, we recommend that systematic reviews define the decisionmaking context, report the sources of evidence used (e.g., estimates of baseline risks or treatment effects), be explicit about if and how patient preferences are considered, and provide a rationale for choosing a particular quantitative approach for comparative assessment of benefits and harms.

Conclusion. Quantitative approaches for comparative assessment of benefits and harms have strengths and limitations. The choice of a particular approach depends on the decisionmaking context, the quality and quantity of available data, and the epidemiological-statistical expertise of the systematic review team. A quantitative approach may help to improve the transparency of a review, relative to a qualitative approach, by being explicit about how benefits and harms are estimated and compared. Such transparency may help decisionmakers give proper consideration to complex information about benefits and harms.”

Read Full Post | Make a Comment ( Comments Off on Methods for Benefit and Harm Assessment in Systematic Reviews – AHRQ – November 2012 )

Liked it here?
Why not try sites on the blogroll...