Research & Policy Questions

Each of the career pathways studies is guided by a set of research questions that is central to the evaluation. Below is a list of the research questions specific to each study.

Pathways for Advancing Careers and Education (PACE) – Short-Term Outcomes

Implementation Study:

  1. What is the basic design and underlying logic of each PACE program? What are its elements? What is its institutional and community context?
  2. To what extent was the intervention delivered as planned? What was the fidelity to the design? If the program’s structure and services changed during the study period, why?
  3. What were the treatment group’s participation patterns and experiences with program services? With transitions beyond the core career pathways program?
  4. What are the differences in type, duration, and content of services received by treatment and control group members?
  5. What are the lessons from PACE programs for the field in the areas of scalability and sustainability?

Impact Study:

  1. What is the impact of each program on key indicators of progress in career pathways‐relevant training, such as persistence in education and the achievement of certificates and degrees?
  2. What are the impacts of each program on entry to career‐track employment, and earnings?
  3. What are the impacts of each program on individual and family well‐being?
  4. Do impacts vary by subgroups or other moderators of impact, and, if so, which characteristics are associated with larger or smaller effects?
  5. Are there impacts on program services and/or intermediate-term outcomes that seem to be acting as pathways to observed impacts on attainment of certificates and degrees, or earnings?

Health Professions Opportunity Grant (HPOG) 1.0 – Short-Term Outcomes

  1. What impacts do the HPOG programs as a group have on the outcomes of participants and their families?
  2. To what extent do those impacts vary across selected subpopulations?
  3. Which locally adopted program components influence average impacts?
  4. To what extent does participation in a particular HPOG component (or components) change the impact on trainees?
  5. What services comparable to those available to HPOG participants exist in the local community?
  6. What is the relative supply of, and accessibility to, those counterfactual services?
  7. What services and trainings did control group members use?

In the follow-up 36-month (intermediate-term outcomes) and 72-month (long-term outcomes) studies, the PACE and HPOG 1.0 studies will address the following questions:

PACE and HPOG 1.0 36-Month Study – Intermediate Outcomes

  1. What are the intermediate-term effects of the HPOG and PACE programs on their populations of interest?
  2. How do effects of career pathways programs vary over time, across outcomes or domains, by occupational sector, by program model, and by participant characteristics?
  3. Do different HPOG and PACE models, strategies, or components lead to different impacts for participants?
  4. How can career pathways models be adjusted to promote intermediate-term outcomes for participants?

PACE and HPOG 1.0 72-Month Study – Long-Term Outcomes

  1. What are the long-term effects of the HPOG and PACE programs on their populations of interest?
  2. How do effects of pathways programs vary over time and across outcomes/domains, by participant characteristic and occupational sector?
  3. Do different HPOG and PACE models, strategies or components lead to different impacts for participants?
  4. How can career pathways models be adjusted to promote long-term outcomes for participants?

Tribal HPOG 1.0 Evaluation

  1. Have grantees incorporated structures necessary to enhance the health care workforce of the community?
  2. Have grantees implemented processes that successfully prepare participants for employment in the tribal health care sector?
  3. Is there evidence that participation in the program resulted in successful employment and workforce capacity-building outcomes?

HPOG 1.0 National Implementation Evaluation (NIE)

  1. How are health professions training programs being implemented across the grantee sites?
  2. What changes to the service delivery system are associated with program implementation?
  3. What individual-level outputs and outcomes occur (e.g., recruitment, enrollment, retention, completion (accreditation/certification), job entry, employment retention and advancement, and earnings)?

HPOG 1.0 Implementation, Systems, and Outcomes (ISO)

  1. How are health professions training programs being implemented across the grantee sites?
  2. What changes to the service delivery system are associated with program implementation?
  3. What individual-level outputs and outcomes occur—e.g., recruitment, enrollment, retention, completion (accreditation/certification), job entry, employment retention and advancement, and earning?
  4. What key components appear necessary to, or contribute to the success of, these programs?

HPOG University Partnerships

Research Goals

  • To examine questions relevant to the HPOG program goals and objectives, as delineated by the legislation
  • To augment the Administration for Children and Families’ (ACF’s) multi-pronged evaluation of HPOG demonstrations by focusing on particular questions relevant to career track education and training programs in the health professions and related labor market issues
  • To support research to inform and improve HPOG program performance
  • To emphasize the importance of developing true working research partnerships with HPOG programs and other relevant entities within the community, thereby increasing the effectiveness of the applied research work
  • To address issues of current relevance to decisionmakers at the local, state, and national levels in order to inform policy decisions and solutions, particularly those related to career track education and training programs and related labor market issues for recipients of Temporary Assistance for Needy Families and other low-income populations, and tribal communities

HPOG 2.0 National Evaluation

  • What is the impact of an offer of an HPOG 2.0 slot on participant educational progress, employment, and earnings?
  • What is the impact of an offer of an HPOG 2.0 slot on receipt of training, financial assistance, child care, financial assistance for child care, and various forms of counseling?
  • Do impacts vary by key participant characteristics?
  • Are some program features and implementation strategies associated with readiness to work or increased earnings in the near or intermediate term?
  • What are the benefits and costs of the HPOG 2.0 Program from the perspectives of program participants; federal, state, and local governments; and society as a whole?

Tribal HPOG 2.0 Evaluation

Structure

  • In what ways was the program designed or modified for tribal organizations?
  • To what degree do the Tribal HPOG 2.0 programs conform to the career pathways framework? What are the pathways?
  • What changes to the service delivery system are associated with program implementation?

Process

  • How are health professions training programs being implemented across the grantee sites?
  • What occupational training opportunities are available to Tribal HPOG 2.0 participants? What is the nature of pre-training, supportive services, job placement, and retention services?
  • Which program components do stakeholders believe to be the most effective in improving outcomes?

Outcomes

  • What are the individual-level outputs and outcomes for participants in the Tribal HPOG 2.0 programs?
  • Do some programs or program components appear to be associated with positive outputs and outcomes for tribal populations? If so, what are the hypothesized reasons for differences between outcomes?
  • Do different program models, strategies, or components appear to lead to different outcomes for participants?
  • Is there evidence that participation in the program is positively associated with successful employment and workforce capacity building outcomes?