APPAM Institutional Member Forum - Bringing Rigor and Intentionality to Mixed Methods Evaluations of Social Programs

hosted by:

Westat 

June 5, 2019 at The Conference Center at Heart House, American College of Cardiology, 2400 N Street Northwest, Washington, DC 20037

 

Mixed methods evaluation seeks to integrate a body of qualitative and quantitative evidence to form conclusions about the success, or impact, of a social policy intervention that neither piece alone—nor even the concatenation of their findings—can produce.  Despite its promise, concern persists that too often the qualitative and quantitative “mixing” occurs only after investigation ends and that mixed methods approaches lack the scientific rigor to be trusted.  This day-long forum sponsored by Westat and APPAM focused researchers on sharpening and expanding the mixed method paradigm to respond to these concerns and to enable impact evaluations to make larger contributions to policy learning.  Major themes included the roles of mixed methods evaluation in explaining/understanding impact findings from randomized control trials and in strengthening quasi-experimental analytic methods to ward off selection bias.  A view of how government agencies can benefit from the approach followed.  Closing perspectives—both skeptical and hopeful—considered where mixed methods evaluation may take us going forward.  Audience participation opportunities interlaced the day.

 

You can read the content highlights via live tweets!

 

The half-day agenda for 200 registrants was as follows:

8:00 am - 9:00 am	Registration & Morning Refreshments
9:00 am - 9:15 am	Welcome  Scott Royal, Westat View the video of this welcome note
9:15 am - 9:25 am	Overview of Day   Patty Troppe, Westat
9:25 am - 9:45 am	Mixed Methods Evaluations: Purpose, Scope, and Challenges   Demetra Nightingale, Urban Institute View the session slides View our interview with Demetra Nightingale View the video of this session
9: 45 am - 10:45 am	Pushing the Envelope of Mixed Methods Evaluation to Learn from Prospective Impact Studies    Virginia Knox, MDRC  View the session slides View video of this session Debra Rog, Westat (Reaction, followed by open discussion) View the session slides View video of this reaction
10:35 am - 11:00 am	Break
11:00 am - 11:45 am	Using Mixed Methods Evaluation to Support Better Non-Experimental Analysis   Stephen Bell, Westat View the session slides Read our interview with Stephen Bell View the video of this session Jeffrey Smith, University of Wisconsin-Madison (Reaction, followed by open discussion) View the session slides View the video of this reaction
11:45 am - 12:30 pm	What Would Benefit Funders and Policymakers?   Maria Woolverton, DHHS/ACF/OPRE  View the video of this session   Marsha Silverberg, ED/IES/NCEE View the session slides View the video of this session
12:30 pm - 1:20 pm	Lunch Service/Announcement of Breakroom Assignments   Patty Troppe, Westat
1:20 pm - 2:20 pm	Sharing of Examples and Real-World Challenges (breakout sessions)   • "Mixed Methods Evaluation for Better Understanding of RCT Results" - Michelle Derr, Mathematica Policy Research   • "Mixed Methods Evaluation to Better Support Non-Experimental Impact Analysis" - Douglas Walton, Abt Associates    • "Policy Questions Besides Intervention Impact Worth Addressing by Mixed Methods" - Irma Perez-Johnson, American Institutes for Research
2:25 pm - 3:10 pm	Reports from Breakout Sessions View video of these reports
3:10 pm - 3:35 pm	Break
3:35 pm - 3:55 pm	Haven't We Been Here Before?  Convince Me   Larry Orr, Johns Hopkins University View the session slides View the video of this session
3:55 pm - 4:15 pm	Potential and Predicted Progress for the Next Decade    Carolyn Hill, MDRC View the session slides View the video of this session
4:15 pm - 4:45 pm	Open Discussion  View the Closing Remarks by Patty Troppe, Westat

 

 

Share your memories from this event on Twitter using the hashtag #APPAMWestat!