How do we know if a program made a difference? A guide to statistical methods for program impact evaluation – 2014

Posted on June 24, 2014. Filed under: Evidence Based Practice, Public Hlth & Hlth Promotion | Tags: , |

How do we know if a program made a difference? A guide to statistical methods for program impact evaluation – 2014

Lance, P., D. Guilkey, A. Hattori and G. Angeles. (2014). How do we know if a program made a difference? A guide to statistical methods for program impact evaluation. Chapel Hill, North Carolina: MEASURE Evaluation.
MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) and the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR).

“Abstract:
This manual provides an overview of core statistical and econometric methods for program impact evaluation (and, more generally, causal modelling). More detailed and advanced than typical brief reviews of the subject, it also strives to be more approachable to a wider range of readers than the advanced theoretical literature on program impact evaluation estimators. It thus forms a bridge between more basic treatments of the essentials of impact evaluation methods and the more advanced discussions. It seeks to discuss impact evaluation estimators in a thorough manner that does justice to their complexity, but in a fashion that is approachable.

The manual is targeted to: public health professionals at programs, government agencies, and NGOs who are the consumers of the information generated by program impact evaluations; professionals serving the aforementioned role in any area of programming that influences human welfare; graduate students in public health, public policy and the social sciences; technical staff at evaluation projects; journalists looking for a more nuanced understanding of the steady stream of impact (and, more broadly, causal) studies on which they are asked to report; analysts at health analytics organizations; and so on.”

Advertisements
Read Full Post | Make a Comment ( Comments Off on How do we know if a program made a difference? A guide to statistical methods for program impact evaluation – 2014 )

Patient centered medical home (PCMH): Evidence and Evaluation – toolbox of methods used to evaluate and refine PCMH models

Posted on April 23, 2013. Filed under: Health Mgmt Policy Planning | Tags: , |

Patient centered medical home (PCMH): Evidence and Evaluation – toolbox of methods used to evaluate and refine PCMH models

“About the Evidence:

Policy decisions concerning the PCMH must rest on sound evidence about whether this model of care helps achieve the Triple Aim of improved patient outcomes, improved patient experience, and improved value. In this section, explore information and resources for PCMH researchers, evaluators, and decision makers.

PCMH Research Methods Series

The PCMH Research Methods Series was commissioned by AHRQ and developed under contract by Mathematica Policy Research, with input from other nationally recognized thought leaders in research methods and PCMH models. The series is designed to “expand the toolbox” of methods used to evaluate and refine PCMH models and other health care interventions. This toolbox of novel and underused methods can equip evaluators and implementers to better assess and refine PCMH models and to meet the evidence needs of PCMH stakeholders more effectively. Each of the briefs describes a method and how PCMH researchers have used it or could do so, discusses advantages and limitations of the methods, and provides resources for researchers to learn more about the method.”

… continues on the site

Read Full Post | Make a Comment ( Comments Off on Patient centered medical home (PCMH): Evidence and Evaluation – toolbox of methods used to evaluate and refine PCMH models )

National Obesity Observatory [NHS] – Standard evaluation frameworks

Posted on October 5, 2012. Filed under: Health Status, Research | Tags: , , |

National Obesity Observatory [NHS] – Standard evaluation frameworks

“NOO has now produced three Standard Evaluation Frameworks (SEFs):

SEF for weight management interventions (published March 2009)
SEF for physical activity interventions (published September 2012)
SEF for dietary interventions (published September 2012)

The aim of the Standard Evaluation Frameworks (SEF) is to support high quality, consistent evaluation of weight management, diet and physical activity interventions in order to increase the evidence base.

The SEFs provide introductory guidance on the principles of evaluation, and list ‘essential’ and ‘desirable’ criteria. Essential criteria are presented as the minimum recommended data for evaluating a weight management intervention. Desirable criteria are additional data that would enhance the evaluation. Supporting guidance describes why particular criteria have been categorised as essential or desirable, and provides further information on collecting data. The SEFs are essential reading to those commissioning, running or evaluating weight management, diet or physical activity interventions.”

Read Full Post | Make a Comment ( Comments Off on National Obesity Observatory [NHS] – Standard evaluation frameworks )

Evaluating disease management programmes. Learning from diverse approaches across Europe – RAND – 2012

Posted on October 5, 2012. Filed under: Chronic Disease Mgmt | Tags: , |

Evaluating disease management programmes. Learning from diverse approaches across Europe – RAND – 2012

“The DISMEVAL consortium (Developing and validating disease management evaluation methods for European healthcare systems, a consortium of ten partners in seven countries led by RAND Europe), examined approaches to chronic disease management in 13 countries across Europe and tested the methods being used to evaluate these in six countries.”

Read Full Post | Make a Comment ( Comments Off on Evaluating disease management programmes. Learning from diverse approaches across Europe – RAND – 2012 )

Progress in Measuring National Well-being outlined – Office of National Statistics [UK] – 24 July 2012

Posted on August 1, 2012. Filed under: Health Status | Tags: , , |

Progress in Measuring National Well-being outlined – Office of National Statistics [UK] – 24 July 2012

“The articles published today are:
Measuring Subjective Well-being in the UK – First Annual ONS Experimental Subjective Well-being Results ;
Measuring National Well-being – Where we Live;
Measuring National Well-being – Health;
Measuring National Well-being – Summary of Proposed Domains and Measures; and Report on the consultation on proposed domains and measures.”

… continues

Read Full Post | Make a Comment ( Comments Off on Progress in Measuring National Well-being outlined – Office of National Statistics [UK] – 24 July 2012 )

Measuring well-being. A guide for practitioners. A short handbook for voluntary organisations and community groups – nef – 30 July 2012

Posted on August 1, 2012. Filed under: Health Status | Tags: , , |

Measuring well-being. A guide for practitioners. A short handbook for voluntary organisations and community groups – nef – 30 July 2012

Publications page

“Executive Summary

This short handbook on measuring well-being is produced by the Centre for Well-being at nef (the new economics foundation) with input from nef consulting. It is designed primarily for voluntary organisations and community groups delivering projects and services, to help them kick-start the process of measuring well-being outcomes.

By measuring the well-being of the people we aim to support, information can be gathered which can be used, for example, to improve the design and delivery of projects and services, to target projects and services at the people who are in most need, to tailor provision to suit needs, and to support funding applications.”

… continues on the site

nef – economics as if people and the planet mattered

Read Full Post | Make a Comment ( Comments Off on Measuring well-being. A guide for practitioners. A short handbook for voluntary organisations and community groups – nef – 30 July 2012 )

A Guide to Evaluation in Health Research – Canadian Institutes of Health Research – June 2012

Posted on June 19, 2012. Filed under: Research | Tags: , |

A Guide to Evaluation in Health Research – Canadian Institutes of Health Research – June 2012

Prepared by: Sarah Bowen, PhD, Associate Professor, Department of Public Health Sciences, School of Public Health, University of Alberta

Read Full Post | Make a Comment ( Comments Off on A Guide to Evaluation in Health Research – Canadian Institutes of Health Research – June 2012 )

National Evaluation of the Department of Health’s Integrated Care Pilots. Final report – RAND Europe, Ernst & Young LLP, prepared for the Department of Health – March 2012

Posted on March 27, 2012. Filed under: Health Mgmt Policy Planning | Tags: , |

National Evaluation of the Department of Health’s Integrated Care Pilots [UK]. Final report – RAND Europe, Ernst & Young LLP, prepared for the Department of Health – March 2012

Key messages
● While much of the wider literature focuses on ‘models’ of integrated care, we found that Integrated Care Pilots (ICPs) developed and implemented a loose collection of ‘integrating activities’ based on local circumstances. Despite the variations across the pilots, a number of aims were shared: bringing care closer to the service user; providing service users with a greater sense of continuity of care; identifying and supporting those with greatest needs; providing more preventive care; and reducing the amount of care provided unnecessarily in hospital settings.
● Most pilots concentrated on horizontal integration – e.g., integration between community-based services such as general practices, community nursing services and social services rather than vertical integration – e.g., between primary care and secondary care.
● Integrated care led to process improvements such as an increase in the use of care plans and the development of new roles for care staff. Staff believed that these process improvements were leading to improvements in care, even if some of the improvements were not yet apparent. A range of other improvements in care were reported by pilots following local evaluations. We have reported these but they lie beyond the scope of the national evaluation.
● Patients did not, in general, share the sense of improvement.” … continues on the site

Read Full Post | Make a Comment ( Comments Off on National Evaluation of the Department of Health’s Integrated Care Pilots. Final report – RAND Europe, Ernst & Young LLP, prepared for the Department of Health – March 2012 )

Auditing Behaviour Change – National Audit Office [UK] – September 2011

Posted on September 28, 2011. Filed under: Public Hlth & Hlth Promotion | Tags: , , |

Auditing Behaviour Change – National Audit Office – September 2011

“Behaviour change is important for value for money because it can often contribute to, or be a prerequisite for achieving a policy outcome cost-effectively. As behaviour change is rapidly becoming more prevalent, reporting on value for money is a necessity. However, due partly to its complexity, doing so can be difficult. The NAO has developed this guide to represent our emerging thinking on what auditors might look for when doing a value for money assessment.”

Read Full Post | Make a Comment ( None so far )

Collection of Resources on Evaluation – CoRE – National Obesity Observatory NHS

Posted on September 27, 2011. Filed under: Public Hlth & Hlth Promotion | Tags: , |

Collection of Resources on Evaluation – CoRE – National Obesity Observatory NHS 

Access the new Evaluation data collection tool

“Evaluation is about judging the value of an activity and assessing whether or not it has achieved what it set out to do. In health promotion, an evaluation determines the extent to which a programme has achieved its objectives, and will assess how different processes contributed to achieving these objectives.

Evaluation is particularly important for interventions that aim to tackle overweight and obesity. We need to know as much as possible about which approaches are likely to yield results, and to do this we need to be able to compare results across settings, populations and types of intervention. We can then focus public health investment appropriately. This is even more important in a time of restricted public finances.

The aim of NOO’s Collection of Resources on Evaluation (CoRE) is to provide information and resources to support practitioners with an interest in the evaluation of interventions related to obesity, overweight, underweight and their determinants. The current version of CoRE covers: NOO’s Standard Evaluation Framework (SEF); evaluation data collection tool (including details of local interventions); other evaluation guidance; reports from evaluation of nationally-initiated schemes; and evaluation websites.

CoRE is divided into the following sections:

Standard Evaluation Framework
Evaluation data collection tool   
Evaluation guidance
Evaluation reports
Evaluation websites “

Read Full Post | Make a Comment ( None so far )

Measuring Teamwork in Health Care Settings: A Review of Survey Instruments – Harvard Business School – Published 22 September 2011

Posted on September 23, 2011. Filed under: Health Professions, Workforce | Tags: , |

Measuring Teamwork in Health Care Settings: A Review of Survey Instruments – Harvard Business School – Published 22 September 2011
Authors: Melissa A. Valentine, Ingrid M. Nembhard, and Amy C. Edmondson

Executive Summary:
It is critical to accurately assess teamwork in health-care organizations. About 60 percent of primary-care practices in the United States use team-based models to coordinate work across the broad spectrum of health professionals needed to deliver quality care; in many other countries the percentage is almost 100 percent. While the benefits of effective teamwork are substantial, effective teamwork is often lacking in these settings, with negative consequences for patients. To date, little has been known about the survey instruments available to measure teamwork. In this paper Valentine, Nembhard, and Edmondson report the results of their systematic review of survey instruments that have been used to measure teamwork in various contexts. Their research helps to identify existing teamwork scales that may be most useful in testing theoretical models. Key concepts include:

•Researchers often develop a new scale for their project rather than adapt existing scales. It would be better to utilize existing, psychometrically valid scales when possible so that cumulative knowledge of teamwork can be built.
•Many scales have been developed to assess teamwork. However, only eight scales satisfy the standard psychometric criteria the authors identified, and only three of those have been significantly associated with non-self-reported outcomes.
•Future research needs to clarify the concept of teamwork. Currently, the variation in ways of conceptualizing teamwork even within the scales that do show relationships to outcomes of interest makes it difficult to know what dimensions are core versus peripheral to the concept.
•The criteria set forth in this article should be considered standard research practice, and as such the scales that the authors identified are worthy of attention.”
..

Read Full Post | Make a Comment ( None so far )

For the Public’s Health: The Role of Measurement in Action and Accountability – Institute of Medicine Consensus Report – 8 December 2010

Posted on February 23, 2011. Filed under: Public Hlth & Hlth Promotion | Tags: , , , |

For the Public’s Health: The Role of Measurement in Action and Accountability – Institute of Medicine Consensus Report – 8 December 2010

“Despite having the costliest medical care delivery system in the world, Americans are not particularly healthy. Recent international comparisons show that life expectancy in the U.S. ranks 49th among all nations, and infant mortality rates are higher in the U.S. than in many far less affluent nations. While these statistics are alarming, the bigger problem is that we do not know how to reverse this trend. Our lack of knowledge is due in large part to significant inadequacies in the system for gathering, analyzing, and communicating health information about the population.

To inform the public health community and all other sectors that contribute to population health, the Robert Wood Johnson Foundation commissioned the IOM to examine three major topics that influence the health of the public—measurement, laws, and funding. In this, the first of three reports, the IOM reviews current approaches for measuring the health of individuals and communities and suggests changes in the processes, tools, and approaches used to gather information about health outcomes and their determinants. The IOM recommends developing an integrated and coordinated system in which all parties—including governmental and private sector partners at all levels—have access to timely and meaningful data to help foster individual and community awareness and action.”

Read Full Post | Make a Comment ( None so far )

Handbook on Impact Evaluation: Quantitative Methods and Practices – International Bank for Reconstruction and Development – 2010

Posted on October 4, 2010. Filed under: Evidence Based Practice, Health Systems Improvement | Tags: , |

Handbook on Impact Evaluation: Quantitative Methods and Practices – International Bank for Reconstruction and Development – 2010

Khandker, S.R., Koolwal, G.B., & Samad, H.A. (2010). Handbook on Impact Evaluation: 
Quantitative Methods and Practices. The International Bank for Reconstruction and Development.

ISBN: 978-0-8213-8028-4
eISBN: 978-0-8213-8029-1
DOI: 10.1596/978-0-8213-8028-4

Extract from the foreword:

“Handbook on Impact Evaluation: Quantitative Methods and Practices makes a valuable contribution in this area by providing, for policy and research audiences, a comprehensive overview of steps in designing and evaluating programs amid uncertain and potentially confounding conditions. It draws from a rapidly expanding and broadbased literature on program evaluation—from monitoring and evaluation approaches to experimental and nonexperimental econometric methods for designing and conducting impact evaluations.”

Read Full Post | Make a Comment ( None so far )

The role of evaluation in evidence-based decision-making – BIS Department for Business Innovation and Skills [UK] – 11 August 2010

Posted on August 28, 2010. Filed under: Health Mgmt Policy Planning | Tags: , |

The role of evaluation in evidence-based decision-making – BIS Department for Business Innovation and Skills [UK] – 11 August 2010

Summary:
This strategy aims to improve the evaluation of the department’s’ policies and to ensure results and lessons learned are integrated into policy making.

Read Full Post | Make a Comment ( None so far )

Liked it here?
Why not try sites on the blogroll...