Child Welfare Effectiveness

by

Melissa Jonson-Reid, Richard P. Barth, Chien-Jen Chiang

• LAST MODIFIED: 23 May 2024
• DOI: 10.1093/obo/9780195389678-0335

Introduction

In the United States, the family is considered the primary unit responsible for keeping children safe and providing adequate care to address health and developmental needs. When a child is harmed or at risk of harm, the child welfare system is charged with intervening. The child welfare system’s goals are to keep a child with their family whenever possible or to provide a safe and permanent option until they can return home or achieve permanency. The child welfare system must screen all referrals, operate with minimal funding, and leverage scarce external resources while balancing family rights and child safety. Evaluating such a system is challenging. States have different definitions of maltreatment and standards for intervention. There are different levels of intervention (from fully voluntary case management to court-mandated services). That means for any given group effectiveness can be measured using varied metrics such as fatality or serious harm, children remaining with their family, placement types, and attention to child health. Others may judge effectiveness in regard to particular values—for example, whether the least possible interference in a family is made, whether children from all identity groups are equally represented across service categories, or whether services are provided at a low cost. Any chosen outcome is complicated when populations served, risk factors, system funding, and policies are dynamic. The question is not if the child welfare system is effective, but what part, according to what metric, for whom, and where? In an era of reform, understanding what we know about these questions is key to improving outcomes for children and families. This chapter reviews key literature organized by system function and priorities from primary prevention through foster care to explore dimensions of effectiveness.

General Overview

The policy and programmatic foundations of child welfare are complex and beyond the scope of this chapter. We refer the reader to other Oxford Bibliographies entries: Child Welfare, Child Welfare Law in the United States, In-Home Child Welfare Services, Differential Response in Child Welfare, Foster Care, Kinship Care, and Tribal Child Welfare Practice in the United States. Testa and Kelly 2020 discusses how the Families First Prevention Services Act of 2018 may alter child welfare practice and outcomes. Additional select resources may offer insight into evaluating child welfare effectiveness. Littell and Shlonsky 2010 suggests the use of Cochrane and Campbell reviews to inform child welfare. Julien-Chinn and Lietz 2019 focuses on how supervisory practices influence uptake of best practices, including consideration of client voice in services. Mildon and Shlonsky 2011 outlines how to integrate implementation science in outcomes research. Jonson-Reid and Chiang 2019 discusses methodological approaches to understanding outcomes in child welfare. Brewsaugh and Prendergast 2022 illustrates how child welfare impact evaluations can meet evidence-based clearinghouse standards. Collins-Camargo, et al. 2011 describes approaches to collaborative multi-site research. Lery, et al. 2022 highlights the importance of understanding client engagement. Chen, et al. 2020 and Henry, et al. 2014 offer examples of different analytic approaches to advance understanding of child welfare.

• Brewsaugh, K., and S. Prendergast. 2022. Ten key design elements for rigorous impact evaluations in child welfare. OPRE Report #2022-171. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, US Department of Health and Human Services.

  This report illustrates strategies for impact evaluations of child welfare that meet evidence-based clearinghouse standards. Elements were derived from the Supporting Evidence Building in Child Welfare project and interviews with clearinghouse representatives.

• Chen, D. G., M. F. Testa, D. Ansong, and K. C. Brevard. 2020. Evidence building and information accumulation: Using the Bayesian paradigm to advance child welfare intervention research. Journal of the Society for Social Work and Research 11.3: 483–498.

  DOI: 10.1086/711376

  This article describes a Bayesian approach to analyzing program outcomes to understand program impact when both formative and summative data exist. The authors discuss the relative advantage of this approach to understand child welfare effectiveness.

• Collins-Camargo, C., K. Shackelford, M. Kelly, and R. Martin-Galijatovic. 2011. Collaborative research in child welfare. Child Welfare 90.2: 69–85.

  This article describes a multi-site collaborative research approach used by the Children’s Bureau Quality Improvement Centers to build the evidence base in child welfare. The article reviews challenges and lessons learned.

• Henry, C., S. Carnochan, and M. J. Austin. 2014. Using qualitative data-mining for practice research in child welfare. Child Welfare 93.6: 7–26.

  Child welfare case file notes are underutilized. This article discusses a qualitative analysis process as a means of capturing practice information in a less intrusive way for the agency.

• Jonson-Reid, M., and C. J. Chiang. 2019. Problems in understanding program efficacy in child welfare. In Re-visioning public health approaches for protecting children. Edited by B. Lonne, D. Scott, D. Higgins, and T. Herrenkohl, 349–377. Cham, Switzerland: Springer Nature.

  This chapter discusses challenges in measuring child welfare outcomes at the policy and program levels. Suggestions capturing usual care services both alone and in conjunction to testing added interventions are provided.

• Julien-Chinn, F. J., and C. A. Lietz. 2019. Building learning cultures in the child welfare workforce. Children and Youth Services Review 99:360–365.

  DOI: 10.1016/j.childyouth.2019.01.023

  This paper highlights the importance of child welfare workforce supervision on the development of a learning culture, which is key to implementing evidence-informed approaches to improving outcomes. The article highlights Family-Centered Practice as an example that also engages client voice.

• Lery, B., K. Malm, and A. McKlindon. 2022. Engagement and persistence in child welfare services: Implications for program effectiveness. OPRE Report #2022-06. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, US Department of Health and Human Services.

  As child welfare leverages external services, this report discusses how stages of engagement from initial consent to retention in services impact assessment of effectiveness.

• Littell, J. H., and A. Shlonsky. 2010. Toward evidence-informed policy and practice in child welfare. Research on Social Work Practice 20.6: 723–725.

  DOI: 10.1177/1049731509347886

  This article outlines how systematic reviews such as Campbell and Cochrane can be used to inform policy and program decision-making. Over-reliance on model adoption may lead to decisions that ignore critical local contexts of child welfare intervention.

• Mildon, R., and A. Shlonsky. 2011. Bridge over troubled water: Using implementation science to facilitate effective services in child welfare. Child Abuse & Neglect 35.9: 753–756.

  DOI: 10.1016/j.chiabu.2011.07.001

  This article highlights developments of implementation science in child welfare. Examples are provided of the need for and use of implementation data to inform effectiveness research.

• Testa, M. F., and D. Kelly. 2020. The evolution of federal child welfare policy through the Family First Prevention Services Act of 2018: Opportunities, barriers, and unintended consequences. Annals of the American Academy of Political and Social Science 692.1: 68–96.

  DOI: 10.1177/0002716220976528

  This paper is one of the first discussions of the FFPSA and is a benchmark that can be used to understand how we arrived at this policy choice and what trends in service delivery and effectiveness are beginning to emerge.