Read Challenges in Measuring Implementation Success text version

Challenges in Measuring Implementation Success

Carol VanDeusen Lukas, EdD

VA Center for Organizational, Leadership & Management (COLMR)

Carmen Hall, RN, PhD

VA Center for Chronic Disease Outcomes Research (CCDOR)

Mariana Gerena-Melia, PhD

VA Center for Organizational, Leadership & Management (COLMR)

3rd Annual NIH Conference on Science of Dissemination and Implementation Methods and Measurement March 15, 2010 Bethesda, MD

Think tank objectives:

Stimulate discussion about terminology and a shared framework

· How do we define implementation success? · Against what shared standards should implementation success be judged?

Explore issues in measuring implementation success

· How do we set the standards for a specific study? · How do we define a complex intervention in operational terms to measure it? · What are appropriate measurement approaches?

2

Van Deusen Lukas, Measuring Implementation Enhancing Implementation Science

1

July 2010 www.queri.research.va.gov/meetings

Implementing a clinical innovation:

Intervention Implementation strategies Clinical innovation Process outcomes Health outcomes

Other factors affecting progress and success

Why do we care?

· Essential to judging effectiveness of strategies for implementing innovative programs

­ Implementation success as dependent variable

· Understanding factors affecting implementation

­ Implementation success as independent variable

4

Van Deusen Lukas, Measuring Implementation Enhancing Implementation Science

2

July 2010 www.queri.research.va.gov/meetings

How do we define implementation success?

· Implementation success is the presence of the innovation as intended in the delivery context · Measuring implementation success is formally

determining that presence

5

What is implementation strategy v. innovation v. outcomes?

· Examples:

Study Bipolar CCM Implementation strategy Replicating effective programs (REP) External facilitation Innovation Collaborative chronic care model (CCM) Organizational model for strengthening implementation of evidence-based practices Process outcome Guideline concordant anti-manic treatment Hand-hygiene compliance

Organizational collaboration for hand-hygiene

· Does this work with your projects?

Van Deusen Lukas, Measuring Implementation Enhancing Implementation Science

3

July 2010 www.queri.research.va.gov/meetings

Does the concept of implementation success apply to implementation processes as well as to the innovation?

Measures of implementation success

? Intervention Implementation strategies Clinical innovation Process outcomes Health outcomes

Other factors affecting progress and success

Against what shared standards should implementation success be judged?

· Fidelity -- Adherence, Scope/reliability,

Intensity/dose, Quality of program, Participant responsiveness, Sustainability · RE-AIM ­ Reach, Effectiveness, Adoption, Implementation, Maintenance

· Are these standards appropriate in all situations? · Are there other standards against which implementation should be judged?

8

Van Deusen Lukas, Measuring Implementation Enhancing Implementation Science

4

July 2010 www.queri.research.va.gov/meetings

How do we set the standards for a specific study?

· Example: Treating tobacco use and dependence

Ask-Advise-Assess-Assist-Arrange

· What have you done? How did you decide which to use?

How do we operationalize a complex intervention to measure it?

· Key elements

­ Important for implementation as well as measurement ­ Replication v tailoring/ customization/ adaptation

· Which features are core to the intervention?

­ Which features have to be present for the intervention to maintain its effectiveness? ­ Which can vary without compromising effectiveness?

10

Van Deusen Lukas, Measuring Implementation Enhancing Implementation Science

5

July 2010 www.queri.research.va.gov/meetings

How do we operationalize...?

· Example: Testing an organizational model to improve hand-hygiene compliance

­ Concept of key elements built into theoretical framework ­ Implementation plan by recipients ­ Ratings by research team

· How have you done it?

What are appropriate measurement approaches?

· Secondary v primary data?

­ Availability of existing data ­ Resource & logistical constraints of collecting new data ­ Qualitative, quantitative

· Who collects?

­ Scale of intervention & role of researcher ­ Expectations/burden of implementation staff collection

12

Van Deusen Lukas, Measuring Implementation Enhancing Implementation Science

6

July 2010 www.queri.research.va.gov/meetings

What measurement approaches are appropriate in what settings?

· Example: ASK · Data sources:

­ electronic records ­ new practice survey ­ chart review ­ coded qualitative data from interviews or documents

What measurement approaches are appropriate in what settings?

· Example: ASK · Levels of analysis:

­ how often an individual patient is asked about tobacco use? ­ number of clinics which implemented a system for universal identification of tobacco use

· How have you measured implementation success?

Van Deusen Lukas, Measuring Implementation Enhancing Implementation Science

7

July 2010 www.queri.research.va.gov/meetings

Recap...

Van Deusen Lukas, Measuring Implementation Enhancing Implementation Science

8

July 2010 www.queri.research.va.gov/meetings

Information

Challenges in Measuring Implementation Success

8 pages

Report File (DMCA)

Our content is added by our users. We aim to remove reported files within 1 working day. Please use this link to notify us:

Report this file as copyright or inappropriate

1088818


Notice: fwrite(): send of 208 bytes failed with errno=104 Connection reset by peer in /home/readbag.com/web/sphinxapi.php on line 531