Tribal Evaluation of the 2nd Generation of HPOG (Tribal HPOG 2.0 Evaluation)

The Tribal HPOG 2.0 evaluation is rigorously assessing the programs administered by the Tribal HPOG 2.0 grantees, using sound scientific methods and grounded in culturally appropriate approaches. The tribal evaluation is using a mixed-methods approach and collecting quantitative and qualitative data from multiple sources. The research questions focus on the Tribal HPOG programs’ structure, processes, and outcomes.

The Health Profession Opportunity Grants (HPOG) Program, administered by the Administration for Children and Families (ACF), U.S. Department of Health & Human Services, supports demonstration projects that provide Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals with the opportunity to obtain education and training for occupations in the healthcare field that pay well and are expected to either experience labor shortages or be in high demand. In 2015, ACF awarded a second round of five-year HPOG grants (referred to as HPOG 2.0) to 32 organizations (including 5 tribal organizations) across 21 states, and concurrently launched separate evaluations of the new tribal and non-tribal programs. The Tribal Evaluation of the 2nd Generation of HPOG (Tribal HPOG 2.0 Evaluation) is a comprehensive implementation and outcome study led by NORC at the University of Chicago under a subcontract with Abt Associates. For more information about the HPOG 2.0 National Evaluation of the non-tribal HPOG 2.0 grantees, see the HPOG 2.0 National Evaluation page.

The tribal evaluation team designed the evaluation in collaboration with the Tribal HPOG 2.0 grantees and the Tribal HPOG 2.0 Technical Work Group, which is comprised of tribal evaluators and subject matter experts, to ensure that the evaluation is using sound scientific methods and grounded in culturally appropriate approaches. The evaluation will provide an in-depth assessment of the HPOG 2.0 programs administered by tribes, tribal organizations, and tribal colleges to 1) identify and assess how programmatic health professions training operations are working; 2) determine differences in approaches being used when programs are serving different sub-populations, including participants with different characteristics and skill levels; and 3) identify programs and practices that seem to be successful in supporting the target population to achieve portable industry-recognized certificates or degrees as well as employment-related outcomes. The questions guiding the evaluation are designed to examine the structures, processes, and outcomes of the Tribal HPOG 2.0 programs. The evaluation will address the following key questions:

Structure

  • In what ways was the program designed or modified for tribal organizations?
  • To what degree do the Tribal HPOG 2.0 programs conform to the career pathways framework? What are the pathways?
  • What changes to the service delivery system are associated with program implementation?

Process

  • How are health professions training programs being implemented across the grantee sites?
  • What occupational training opportunities are available to Tribal HPOG 2.0 participants? What is the nature of pre-training, supportive services, job placement, and retention services?
  • Which program components do stakeholders believe to be the most effective in improving outcomes?

Outcomes

  • What are the individual-level outputs and outcomes for participants in the Tribal HPOG 2.0 programs?
  • Do some programs or program components appear to be associated with positive outputs and outcomes for tribal populations? If so, what are the hypothesized reasons for differences between outcomes?
  • Do different program models, strategies, or components appear to lead to different outcomes for participants?
  • Is there evidence that participation in the program is positively associated with successful employment and workforce capacity building outcomes?

The tribal evaluation team is using both qualitative and quantitative methods to address these evaluation questions. The tribal evaluation team will review program documents, such as grantee applications, semi-annual reports, administrative documents, and program literature and curricula. The evaluation team will also conduct annual site visits that will include in-person interviews with grantee and partner administrative staff (e.g., project directors and managers), program implementation staff (e.g., instructors, service providers) and local employers. In addition, the evaluation team will conduct focus groups with current program participants and interviews with program completers and non-completers in-person during the site visit or via telephone. Qualitative data analysis will identify common themes across the programs, including facilitators and barriers to program implementation and participant outcomes. Additionally, the tribal evaluation team will analyze the participant data collected through the HPOG 2.0 web-based management information system (i.e., the Participant Accomplishment and Grantee Evaluation System (PAGES)) and, when available, wage and employment information through the National Directory of New Hires (NDNH). Data collection for the Tribal HPOG 2.0 Evaluation began in 2017.

HPOG Grantees Participating in the Tribal HPOG 2.0 Evaluation

  • Cankdeska Cikana Community College
  • Turtle Mountain Community College
  • Cook Inlet Tribal Council
  • Ute Mountain Ute Tribe
  • Great Plains Tribal Chairmen’s Health Board

FAQs

Tribal HPOG 2.0 Evaluation Partners and Sponsors

Primary Evaluation Sponsor

OPREThe Tribal HPOG 2.0 evaluation is funded by The Office of Planning, Research and Evaluation (OPRE). OPRE is the principal advisor to the Assistant Secretary for Children and Families, U.S. Department of Health and Human Services, on increasing the effectiveness and efficiency of programs designed to improve the economic and social well-being of children and families.

Lead Evaluator

Abt Associates logoAbt Associates is a mission-driven, global leader in research and program implementation in the fields of health, social and environmental policy, and international development. Known for its rigorous approach to solving complex challenges, Abt Associates is regularly ranked as one of the top 20 global research firms and one of the top 40 international development innovators. The company has multiple offices in the U.S. and program offices in more than 40 countries.

Evaluation Partners

NORC at the University of ChicagoNORC at the University of Chicago is an independent research institution that delivers reliable data and rigorous analysis to guide critical programmatic, business, and policy decisions. NORC collaborates with government agencies, foundations, educational institutions, nonprofit organizations, and businesses to develop enduring knowledge that supports evidence-based decisions.

Additional Links

Contacts

Tribal HPOG Evaluation Project Director
Michael Meit
Senior Fellow
NORC at the University of Chicago
(301) 634-9324
meit-michael@norc.org

Federal Project Officer
Amelia Popham
Senior Social Science Research Analyst
Office of Planning, Research, and Evaluation
Administration for Children and Families
Amelia.Popham@acf.hhs.gov

Hilary C. Forster
Senior Social Science Research Analyst
Team Lead for the Health Profession Opportunity Grants Evaluation Portfolio
Office of Planning, Research, and Evaluation
Administration for Children and Families
hilary.forster@acf.hhs.gov

Other ACF-Sponsored Studies