0915-0061 - BHW Performance Collection Supporting Statement A 03082024

Supporting Statement A

Revision for the Bureau of Health Workforce (BHW) Performance Report for Grants and Cooperative Agreements

OMB Control No. 0915-0061

Revision

Note: HRSA requests a change in the OMB control number to 0906-XXXX, since 0915 is a prefix for HSA (the old name for HRSA).

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

This request is for Office of Management and Budget (OMB) approval of a revision package to continue data collection activities with current and prospective health professions awardees of the Health Resources and Services Administration’s (HRSA’s) Bureau of Health Workforce (BHW). The current approval (OMB #0915-0061) expires on 01-31-2025 and covers data collection efforts through progress reports, as well as annual performance reports for grants and cooperative agreements (PRGCA).

BHW seeks approval from OMB to continue these efforts over the next 3 years with this package. The supporting statement for this request has been updated and contains discussion about the utility of data collected during July 1, 2024, through June 30, 2027; lessons learned from data collection efforts during this time; as well as details regarding BHW’s proposed strategies for reducing the overall burden associated with its data collecting efforts (i.e., progress reports and the PRGCA).

HRSA is obligated to collect performance information on its grants and cooperative agreements as mandated in the Government Performance and Results Act of 1993 (GPRA),¹ the GPRA Modernization Act of 2010 (GPRAMA),² and the Foundations for Evidence-Based Policymaking Act of 2018 requirements.³ The health professions grant programs are governed by the Public Health Service Act (42 U.S.C. 201 et seq.), specifically Titles III, VII, and VIII, which specify additional reporting requirements that are specific to certain programs.

BHW seeks early approval from OMB to account for the following: 1) changes in administration and agency priorities, such as the increased emphasis on community partnerships; 2) for the implementation of new technology, such as Excel uploads to reduce burden; and 3) for changes in programs, such as the implementation of new apprenticeships programs. BHW plans to continue with its current performance management strategy and make only minor changes that reduce burden, simplify reporting, and reflect new Department of Health and Human Services and HRSA priorities, as well as elements that enable longitudinal analysis of program performance.

To reduce burden, in Academic Year (AY) 2022-2023, a new Excel upload feature was implemented for all programs. In addition, BHW proposes the following updates to the currently approved package. BHW proposes new questions to understand the primary and secondary purposes of the partnerships associated with funding. The new, standardized partnership questions appear on the following forms: Program Characteristics (PC) forms PC-1 (Columns 5a and 5c), PC-2 (Columns 5a and 5c), PC-3 (Columns 8a and 8c), PC-4 (Columns 3a and 3c), PC-5 (Columns 3a and 3c), PC-6 (Columns 2a and 2c), PC-7 (Columns 2 and 2c), PC-8 (Columns 3a and 3c), PC-10 (Columns 3a and 3c), and (Experiential Characteristics (EXP) forms EXP-1 (Columns 5a and 5c). Additionally, the existing partnership questions were renumbered and relabeled to ensure consistency and accuracy of reporting. Another proposed update is to Employment-related questions (primary program outcomes). The proposal is to standardize these questions across programs and forms to provide consistent outcomes on employment location, type of employment, and hiring organization. These questions appear on Individual Characteristics (INDGEN) form (Columns 56a, 56b, and 64) and on Individual Prior Year (INDGEN-PY) form (Columns 18a, 18b, 18c). BHW proposes to add Gender questions to the Legislatively Required (LR) form LR-2 in accordance with the guidance from Executive Order 14075 and multiple grantee requests: LR-2 (Columns 12a, 13, 14, 15, 16, 16a, and 17. BHW proposes the addition of a new question to identify residents and fellows funded through supplemental funding streams and greatly reduce burden for awardees INDGEN (2b). This question prevents awardees from having to complete multiple duplicative Annual Performance Reports (APRs). Finally, BHW requests approval to add new questions to measure the outcomes associated with new Apprenticeship programs. Specifically, questions were added to measure additional employment outcomes including the apprentice’s role at the employment site, vulnerable populations served, program satisfaction, and types of competencies graduates were ready to perform. The apprenticeship questions appear in the following forms: INDGEN (Columns 39a, 39b, 56c, 56d, 93- 108) and INDGEN-PY (Columns 21 and 22). See Appendix C for additional detail. Changes detailed here only apply to programs that utilize the applicable forms and columns listed.

Copies of all performance measures and forms are included in this package. No program completes all forms or items, and program awardees only see forms and items specific to them when completing their PRGCA. All measures, forms, and program-specific manuals are located on the HRSA website at https://bhw.hrsa.gov/grants/reportonyourgrant.

2. Purpose and Use of Information Collection

BHW is statutorily tasked with responding to issues specific to the training and supply of the current and future U.S. healthcare workforce (see 42 USC 292 et seq). Currently, BHW funds over 50 different health professions training and loan repayment programs that aim to increase the supply and distribution of the current and future U.S. healthcare workforce. Generally, these programs fall into three distinct categories:⁴

• Infrastructure: refers to programs that are designed to enhance the scope and/or quality of health professions training programs. These programs do not provide direct financial support to students; rather, awardees use funds in a variety of ways including enhancing curriculum and clinical training opportunities, as well as offering faculty development opportunities.

• Direct Financial Support: refers to programs that are designed to provide students of health professions training programs with a financial award to cover costs associated with tuition and/or allowable living expenses. Depending on the nature of the program, awardees of these programs provide scholarships, stipends, or loans to students pursuing health profession-related training or degrees.

• Multipurpose or Hybrid programs: refers to programs that, in accordance with their authorizing statute, may fund a variety of activities to include enhancing training infrastructure, providing direct financial support to health professions students, or supporting enhancements to clinical rotations and training.

In order to carry out its functions, BHW has historically collected data from funded awardees at two specific phases of a grant cycle:

• Phase I: Mid-Year Progress Reports

• Data collected in the form of progress reports serve as the official record of communication between government project officers and awardees and highlight awardees' successes and challenges in meeting the goals of each program. Information provided through progress reports are reviewed by government project officers in BHW and are used to determine progress toward implementing required grant activities; as well as technical assistance needs. In addition, information provided through progress reports also assists BHW in understanding fluctuations in program outcomes reported through the PRGCA.

• This request seeks approval to continue collecting information through progress reports from BHW-funded awardees on an annual basis (Table 2). Submission of progress reports will not coincide with the submission of the PRGCA and will afford government project officers and awardees an additional opportunity for dialogue regarding progress toward program requirements and goals, as well as to respond in a timelier fashion to technical assistance needs. Measures to be used in progress reports can be found in Appendix B.

Table 2.

	Performance Period	Progress Report Due Date
Reporting Schedule	July 1 through February 28	March

*Actual performance period and due date will vary, but it is typically the second week of March.

• Phase II: End-of-Year Annual Performance Reports

• Data collected through the PRGCA serve a number of critical functions and are essential for responding to Federal reporting requirements (e.g., GPRAMA and Evidence Act), understanding emerging issues in the health professions, ensuring compliance with grant and statutory requirements, strengthening overall program performance, and responding to congressional and public inquiries regarding outcomes associated with health professions training and loan repayment programs. For example:

• Providing key metrics for the performance budget. The PRGCA provides all the metrics for BHW’s performance budget that is included in HRSA’s Congressional Justification. BHW currently has 47 GPRA performance budget measures and over 25 additional output measures that are all calculated based on PRGCA data. Additionally, every program in BHW has a program accomplishments paragraph included that is also based on the metrics and data provided by awardees in the PRGCA. Many of the metrics are included in the Health Professions Training Programs dashboard that went live on the HRSA website in 2019 and is updated annually - https://data.hrsa.gov/topics/health-workforce/training-programs.

• Informing program management decisions. In 2021, a new Grantee Scorecard went live within HRSA's Electronic Handbook (EHB) system which streamlines the program management process. It visually displays data from a variety of locations within EHB in one place allowing Project Officers, National Center for Health Workforce Analysis (NCHWA) scientists, and BHW leaders to quickly review important information on awardees. The Scorecard includes key performance metrics, submission timeliness information, and other awardee monitoring details from Academic Year 2018 to present. Visualizations allow users to quickly see whether performance metric results are increasing, decreasing, or staying the same year to year. Another important feature on these dashboards is the inclusion of targets. This allows users to see how a particular awardee, grant program, or portfolio of grant programs are performing as compared to program- and BHW-level targets. Combined, this information is intended to inform program management decisions. The scorecards are also available to awardees so they can see how their specific grant program(s) are doing as compared to the program- and BHW-level targets and make adjustment, as needed.

• Enhancing the agency's understanding about the distribution of individuals receiving direct financial assistance. As a result of the performance measures being collected at the individual level, BHW has been able to more accurately compare training distribution rates across its programs. In addition, we are able to identify a student’s intent to practice in an underserved area and to collect counts/percentages of graduates who are actually practicing in underserved, rural, and primary care areas. BHW is also collecting National Provider Identifier (NPI) data to assist in efforts related to the distribution of graduates of the health professions programs. This effort is helping the Bureau produce stronger outcome data for its programs.

In an effort to implement a stronger performance management strategy throughout the Bureau, BHW changed to an annual PRGCA reporting schedule (see Table 3) in 2016 from a semi-annual reporting schedule. This annual reporting system has served the Bureau well and continues to be a strong performance management strategy with less burden on awardees compared with the semi-annual reporting. BHW will continue to utilize technical assistance calls, quarterly calls, and progress reports to proactively and systematically respond to program performance throughout the fiscal year. In addition, BHW implemented a process called Rapid Cycle Quality Improvement (RCQI) in their Funding Opportunities. The RCQI process requires applicants to use quality improvement techniques to ensure grant activities achieve their intended purposes and promote continuous assessment and improvement of activities as needed over the grant periods. Measures to be used in the PRGCA can be found in Appendix C and are presented separately for each BHW-funded program.

Table 3.

	Performance Period	PRGCA Due Date
Current Annual Reporting Schedule	July 1 through June 30	July 31

3. Use of Improved Information Technology and Burden Reduction

Consistent with the previous reporting cycle, BHW will continue to use the EHB as the portal for data collection. The EHB serves as the system of record for HRSA’s grants and cooperative agreements. Several of the forms have the option to update previous information reported—reducing the need to re-report information which does not vary during the life of a specific grant. For example, demographic information about individuals receiving direct financial support (e.g., stipends, loans, or scholarships) will only have to be reported once. Using awardee-developed unique identifiers, each awardee will only be required to update specific fields—such as financial award amounts, attrition status, graduation status, and 1-year follow-up.

To reduce the reporting burden on high volume awardees who are required to report individual level data on a large number of trainees, BHW implemented an Excel upload several years ago. The Excel upload allows awardees to collect individual-level trainee data (consisting of the trainee’s unique ID, training program, demographic information, aspects of their training, and employment information upon completion of training) and complete a bulk data upload directly into EHB rather than keying in every required data field within EHB. The INDGEN form (see Appendix C for the form) is the only form that has the Excel upload feature. Use of the spreadsheet is voluntary for the HRSA awardee, and the spreadsheet captures the same information as required within EHB on the INDGEN form. Awardees that prefer to enter their data directly into EHB may continue to do so. The Excel upload is essentially a voluntary and alternative input pathway for an awardee’s INDGEN form data. This upload feature significantly reduces awardee burden while allowing HRSA to more effectively evaluate its programs. So far, feedback HRSA received has been unanimously positive. Initially, only the Children’s Hospital Graduate Medical Education Program received this option, as that program has the largest number of INDGEN records. In 2023, BHW successfully expanded this option to all grant programs, allowing all grant programs to benefit from this timesaving, burden-reducing feature.

BHW’s ability to follow trainees after the completion of their training to find out if they are employed in health care and/or work in underserved areas is critical to evaluate the effectiveness and success of BHW health professions programs. Section 5103 of P.L. 111-148 requires a longitudinal evaluation of individuals who have received education, training, or financial assistance from programs funded through the Bureau. In addition, the GPRAMA provides a stronger, more precise framework for performance management within the federal government. In accordance with this law, and as directed by the Office of Management and Budget and most recently the Evidence Act of 2018, federal agencies must place a stronger emphasis on performance management and evaluation activities to help support their annual budget requests and demonstrate to the public the effectiveness of their programs. As part of a larger performance management framework, BHW is committed to longitudinal evaluation of its programs to gain better understanding of factors associated with recruiting, retaining, and diversifying the healthcare workforce. The results of this effort will be used to inform the continued offering of current BHW programs and the development of future programs.

Two additional technological advancements were implemented in 2023. BHW proactively updated the EHB system and the PRGCA report to ensure compatibility with Mozilla Firefox, Google Chrome, and Microsoft Edge. Previously, it was only compatible with Internet Explorer, but awardees have long requested additional browser options. This upgrade prevented awardees from having to download additional browsers to complete their reports and ensured a smoother reporting cycle.

BHW also implemented an address standardization feature for the EXP form. Clinical training site addresses that are entered into the EXP form by awardees are checked against the service objects locator. The service objects locator returns standardized addresses. Awardees can review and save the updated address data in their EXP form. This new feature will greatly improve data accuracy and will reduce awardee burden by preventing them from having to look up ZIP Code extensions and manually enter them into the system.

4. Efforts to Identify Duplication and Use of Similar Information

BHW has engaged in a thorough analysis of its tools to identify redundancy and/or duplication of measures across its various data collection activities. Below are summaries of strategies used with each data collection activities to eliminate duplication and reduce burden.

Reducing duplication and burden associated with progress reports

Over the last 8 years, BHW has used the revised progress report, and this has provided programs with valuable information and has streamlined reporting for awardees. In both 2012 and 2015, BHW eliminated the duplication of measures between the progress reports and the PRGCA. Awardees and government project officers have been very pleased with this reduction in burden and progress reports are being used more frequently to assess program performance. The progress report in Appendix B has not been changed since the last OMB submission. It does not contain any duplicative measures and it focuses on assessing activities implemented, achievements and barriers encountered for each activity, and technical assistance needs of awardees.

Reducing duplication and burden associated with the PRGCA

Based on feedback from awardees, staff, evaluators, and public comment, BHW continues to revise the PRGCA to eliminate duplicative efforts. Migrating from semi-annual to annual reporting significantly reduced burden on awardees, government project officers, and data scientists in NCHWA. In addition, technological enhancements have been made to the reporting system where values are automatically totaled for the awardee. In the past, the awardees had to enter individual values and totals, but with the enhancements the system automatically totals and auto-populates cells for the awardees. Similarly, on forms that request location data, city and state are auto-populated after a ZIP Code is entered. This improves data accuracy and reduces awardee burden of having to manually enter city and state information for records.

One of the major changes in 2019 was the ability to have the awardees only report individual-level data instead of also reporting aggregate data. NCHWA scientists calculate aggregate-level estimates for each program removing that burden from the awardee. This process automatically reduces the number of tables required from the awardees of direct financial support programs to complete thus significantly reducing burden for the awardees and government project officers who are required to review all the forms submitted by awardees. Additionally, the individual level data helps the Bureau achieve its longitudinal evaluation plans as collection of this data allows BHW to determine whether a graduate of our programs participated in one or more of our federally funded programs, is currently practicing in primary care, is currently practicing in a rural and/or underserved community, and the primary discipline under which they are certified or licensed. This information also helps BHW use data-driven strategies for implementing its programs as programs can focus on areas and disciplines of short supply as well as helping its graduates find employment in underserved areas, all top priorities for HRSA and BHW.

Similarly, in 2019 BHW conducted an analysis to identify other areas of duplication among forms that capture program-level characteristics for infrastructure and multipurpose or hybrid programs. BHW revised the breadth and depth of measures in each form to ensure that only measures that are most salient to program management and performance reporting are captured in a manner that is appropriate to the purpose, design and impact of each program. BHW has continued this process of removing unnecessary measures from individual grant programs. This is done each year to streamline reporting and reduce burden. Appendix C contains measures for each of BHW's health professions training and loan repayment programs.

In 2022, BHW undertook a large project to streamline reporting of professions and discipline data. BHW split the profession/discipline questions into two parts to simplify the reporting. The selection options were lengthy and difficult to navigate, and awardees reported that it was time consuming to look for specific options. After splitting the profession/discipline questions into two parts, and standardizing the options across INDGEN, EXP, Curriculum Development and Enhancement (CDE), Continuing Education (CE), and Faculty Development (FD) forms, BHW effectively reduced redundancy and therefore the time necessary to search for the appropriate professions and disciplines.

I5. Impact on Small Businesses or Other Small Entities

This project does not involve small businesses or other small entities.

6. Consequences of Collecting the Information Less Frequently

Progress Reports

Data collected in the form of progress reports is a key element of BHW's performance management strategy and serves as an official record of communication between government project officers and awardees. These data provide time-sensitive information about the successes and challenges encountered by awardees in implementing required activities. Progress reports also serve as an instrument for determining awardee-specific technical assistance needs. Collecting data annually allows BHW to provide a timely response to awardee-specific concerns and technical assistance needs, as well as respond to emerging issues across the health professions. Annual data collection is the minimum allowed under GPRA.

Performance Reports for Grants and Cooperative Agreements (PRGCA)

Since 2016, data have been collected from awardees on an annual basis to meet federal reporting requirements, respond to congressional inquiries, and strengthen program performance. The implementation of an annual reporting schedule for PRGCA was a critical step in improving BHW’s performance management strategy and reducing awardee burden across the bureau. Analysis of annual vs. semi-annual reporting showed that collection of data on a semi-annual basis imposed additional burden on awardees, government project officers and NCHWA scientists. Awardees and government have all responded very positively to the annual reporting. In short, the move to annual reporting reduced burden, cost, and showed no difficulty with performance management. In fact, data from annual reporting has shown to be more accurate as it aligns with the Academic Year that all our awardees work on.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

The proposed data collection is consistent with guidelines set forth in 5 CFR 1320.5(d) (2).

8. Comments in Response to the Federal Register Notice/Outside Consultation

Section 8A:

A 60-day Federal Register Notice was published in the Federal Register on October 19, 2023, vol. 88, No. 201; pg. 72086. There was one public comment. The commenter was complementary of BHW’s efforts to consolidate performance data into one collection and raised questions related to collaborating with other Departments on the data collection, defining apprenticeships, and making individual-level data publicly available. Specifically, the commenter asked how HRSA collaborates on data collection with agencies outside of the Department of Health and Human services. Our response to the commenter explained the data collected via this OMB package are performance metrics specific to HRSA grant programs and the data are used to meet obligations for performance budgeting, since the authorization to collect government reporting requirements. The commenter also asked how HRSA defines Apprenticeships and how it aligns with definitions from other Agencies. HRSA responded that it uses the Department of Labor’s (DOL) definition of Apprentices and that it included questions from the DOL’s Employment and Training Administration instrument to reduce reporting burden and made data comparable across the agencies. Lastly, the commenter requested that HRSA make more individual-level data available, but statute prohibits HRSA from doing so (see 42 USC 292 et seq.). There was a follow-up comment from the same commenter regarding HRSA working with external researchers to analyze data on workforce program participants. Our response was that HRSA does not work with external researchers to analyze data collected on workforce programs, since statute prohibits HRSA from sharing individual-level data.

A 30-day Federal Register Notice was published in the Federal Register on January 9, 2024, vol 89, No. 6; pg. 1103. There was one public comment. The commenter requested HRSA explain how this data collection helps meet HHS strategic goals and encourages expanded collaboration with other agencies on workforce data. HRSA responded that the information collection request is authorized for the sole purpose of performance budgeting specific to programs within HRSA’s budget. HRSA does not collaborate with outside agencies in this endeavor as their budget lines are distinct from HRSA’s. Furthermore, HRSA is prohibited by statute from sharing individual-level performance data to any outside entity including other government agencies (see 42 USC 292 et seq.). The response also highlights where performance measurement data and data collection instruments can be found on the HRSA website.

Section 8B:

In developing the proposed updates to BHW's data collection activities, scientists from BHW's NCHWA met with government project officers in BHW to discuss updates to the measures, as well as any programmatic changes that were necessary. Government project officers provided critical feedback that assisted NCHWA in updating measurement activities, as well as reducing redundancy and burden. Government project officers and awardees have been extraordinarily pleased with the change to an annual the reporting schedule.

During the evaluation and questionnaire development, BHW consulted with two subject matter experts from HRSA and two directors from HRSA’s health workforce research centers on the topics of content, frequency of collection, and clarity of instructions. In collaboration with government project officers, NCHWA staff held technical assistance sessions to discuss updates to performance reporting. Overall, awardees responded very positively to BHW's efforts to update measurement activities and reduce burden and redundancy.

9. Explanation of any Payment/Gift to Respondents

Respondents will not receive any payments or gifts. Data collection activities are required as part of the grant or cooperative agreement with awardees and are authorized under 45 CFR Part 74.

10. Assurance of Confidentiality Provided to Respondents

Data will be kept private to the extent allowed by law. All data collected by BHW awardees (i.e., program level and/or individual level) will be reported through BHW’s PRGCA system that is built on a secure web-based enterprise framework. Program-level data reported by BHW awardees are aggregate in nature. Individual-level data reported by awardees are de-identified by the awardee and reported to BHW using awardee-specific unique identifiers. To ensure confidentiality, awardees are not asked or required to provide a list that corresponds unique identifiers with actual student names; rather, data is reported and will always remain de-identified. For programs that also report NPI numbers on individual-level data, NPI numbers are publicly available via the Centers for Medicare & Medicaid Services (CMS) and when reported to HRSA, are stored on the EHB system that is equipped to handle this level of PII. EHB uses role-based permissions to further protect the data. Data can only be downloaded for analysis by NCHWA Administrative users. Once downloaded, data is saved on secure sites that are only accessible using two-factor authentication by qualified NCHWA scientists. In situations where BHW has contracted evaluations, data use and non-disclosure agreements are signed by the contractors and BHW. The terms of these agreements provide strict guidelines for data storage and use. Data must be stored in a secure location which is only accessible by those who signed the data use agreement. Access is revoked and data is destroyed immediately upon termination of employment and/or the contract. In accordance with the Health Professions data statute (42 USC 295k et seq.), raw data files are never shared.

11. Justification for Sensitive Questions

Data collection efforts through progress reports and the PRGCA do not obtain information of a sensitive nature. Demographic-related data (e.g., race, ethnicity, age, and gender) are collected in accordance with standards authorized under Section 4302 of the Patient Protection and Affordable Care Act. Veteran status will be measured in a manner that is consistent with the Veteran’s Administration while disadvantaged status will continue to capture financial disadvantaged status, as well as educational disadvantaged status.

12. Estimates of Annualized Hour and Cost Burden

The estimated annualized burden for the proposed data collection activities vary by activity, as well as the types of awardees providing the required information. The estimated number of respondents is the total number of active awardees expected to complete the progress report and required forms for their program. Awardees must complete their progress report and required forms once per academic year. The average burden per response reflects that some forms are more complex and will take more time than other forms. Tables 4a-4d summarize the estimated burden by fiscal year, data collection activity, and type of grant program. The burden table lists out the types of grant programs (infrastructure, hybrid, direct financial support) rather than different forms/instruments. This is done to show the differences in burden by program type. The specific forms and questions used by each program are outlined in Appendix C via a program mapping document and a full list of performance metrics.

12A. Estimated Annualized Burden Hours

Please note, there are minor differences in the burden estimates between the Federal Register notice and the Supporting Statement due to rounding. In this Supporting Statement, you will note that each row represents a specific form within the overall tool. Each form has a different number of respondents and a different average burden. Thus, when we multiply across the row and sum down the column, we get a total that is slightly different than what we get when we plug the info into the simplified table in the Federal Register notice. However, the numbers input here match what is in ROCIS and represent our burden hour request to OMB.

Table 4a. Response for Awardees of Direct Financial Support Programs

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Awardee (Direct Financial Support Program)	Training Program Form	472*	1	.15	70.8
Awardee (Direct Financial Support Program)	Program Characteristics Form	472*	1	.50	236.0
Awardee (Direct Financial Support Program)	IND-GEN	619	1	1	619.0
Awardee (Direct Financial Support Program)	EXP	434*	1	.50	217.0
Awardee (Direct Financial Support Program)	Curriculum Development & Enhancement Form	223*	1	.25	55.8
Awardee (Direct Financial Support Program)	Program Curriculum Changes	65*	1	.50	32.5
Awardee (Direct Financial Support Program)	Hospital Data	65*	1	.50	32.5
Awardee (Direct Financial Support Program)	Faculty Development, Instruction & Recruitment Form	211*	1	.50	105.5
Awardee (Direct Financial Support Program)	Progress Report	569*	1	.50	284.5
Awardee (Direct Financial Support Program)	Grant Purpose Form	88*	1	0.15	13.2
SUB-TOTAL		619			1,666.8

*Note: Total number of respondents for Awardee Direct Financial Support Programs is 619; however, not all awardees are required to complete all forms due to the nature and purpose of their programs. Number of respondents may be equal to or less than 619 for any form. The completion of all required forms is considered a response to this data collection activity.

Table 4b. Response for Awardees of Infrastructure Programs

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Awardee (Infrastructure Program)	Grant Purpose Form	49*	1	.15	7.4
Awardee (Infrastructure Program)	Training Program Form	219	1	.15	32.9
Awardee (Infrastructure Program)	Program Characteristics Form	219	1	.50	109.5
Awardee (Infrastructure Program)	LR-1	106*	1	.25	26.5
Awardee (Infrastructure Program)	LR-2	106*	1	.25	26.5
Awardee (Infrastructure Program)	DV-1	106*	1	.25	26.5
Awardee (Infrastructure Program)	DV-2	106*	1	.25	26.5
Awardee (Infrastructure Program)	DV-3	106*	1	.25	26.5
Awardee (Infrastructure Program)	EXP	219	1	.50	109.5
Awardee (Infrastructure Program)	CE	118*	1	3	354.0
Awardee (Infrastructure Program)	Curriculum Development & Enhancement Form	219	1	.25	54.8
Awardee (Infrastructure Program)	Faculty Development, Instruction & Recruitment Form	170*	1	.50	85.0
Awardee (Infrastructure Program)	Progress Report	177*	1	.50	88.5
Awardee (Infrastructure Program)	IND-GEN	162*	1	.50	81.0
SUB-TOTAL		219			1,055.0

*Note: Total number of respondents for Awardee Infrastructure Programs is 219; however, not all awardees are required to complete all forms due to the nature and purpose of their programs. Number of respondents may be equal to or less than 219 for any form. The completion of all required forms is considered a response to this data collection activity.

Table 4c. Response for Awardees of Multipurpose/Hybrid Programs

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Awardee (Multipurpose or Hybrid Program)	Grant Purpose Form	383*	1	.15	57.5
Awardee (Multipurpose or Hybrid Program)	Training Program Form	901*	1	.15	135.2
Awardee (Multipurpose or Hybrid Program)	Program Characteristics Form	949*	1	.50	474.5
Awardee (Multipurpose or Hybrid Program)	LR-1	218*	1	.25	54.5
Awardee (Multipurpose or Hybrid Program)	LR-2	218*	1	.25	54.5
Awardee (Multipurpose or Hybrid Program)	DV-1	218*	1	.25	54.5
Awardee (Multipurpose or Hybrid Program)	DV-2	218*	1	.25	54.5
Awardee (Multipurpose or Hybrid Program)	DV-3	218*	1	.25	54.5
Awardee (Multipurpose or Hybrid Program)	IND-GEN	990	1	.50	495.0
Awardee (Multipurpose or Hybrid Program)	EXP	983*	1	.50	491.5
Awardee (Multipurpose or Hybrid Program)	CE	365*	1	.25	91.3
Awardee (Multipurpose or Hybrid Program)	Curriculum Development & Enhancement Form	955*	1	.25	238.8
Awardee (Multipurpose or Hybrid Program)	Faculty Development, Instruction & Recruitment Form	846*	1	.50	423.0
Awardee (Multipurpose or Hybrid Program)	State Oral Health Activities	39*	1	.50	19.5
Awardee (Multipurpose or Hybrid Program)	Progress Report	990	1	.50	495.0
Awardee (Multipurpose or Hybrid Program)	State Primary Care Offices	54*	1	.25	13.5
SUB-TOTAL		1,044			3,207.1

*Note: Total number of respondents for Awardee Multipurpose/Hybrid Programs is 1,044; however, not all awardees are required to complete all forms due to the nature and purpose of their programs. Number of respondents may be equal to or less than 1,044 for any form. The completion of all required forms is considered a response to this data collection activity.

Table 4d. Response for Awardees of All Program Types

	Number of Respondents	Number of Responses per Respondent	Total Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Total	1,882	1	1,882	3.150318809776833	5,928.9

12B.

Based on the estimated total number of burden hours, it is estimated that the annualized cost to respondents is approximately $355,847 (Table 5). This result was obtained by multiplying the number of burden hours by the average hourly wage rate of an individual employed in an academic setting. This labor category was used because the general academic setting category will capture the multiple professions who might fill out these forms. (Note: Wage rates were obtained from the Department of Labor. Average Hourly Rate for this labor category is $30.13, as of August 2023. Wage has been doubled to account for overhead costs. Median hourly rates are not currently available). Data collection and reporting activities are a grant requirement authorized under 45 CFR Part 74.

Table 5. Estimated Annualized Burden Costs by Awardee Program

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Awardee (Direct Financial Support Program)	1,666.8	60.26	$100,038.34
Awardee (Infrastructure Program)	1,055.0	60.26	$63,318.10
Awardee (Multipurpose/Hybrid Program)	3,207.1	60.26	$192,490.14
Total	5,928.9		$355,846.58

(Hourly rate determined using Labor Category ID CES6500000008, Education and Health Services, August 2023).

13. Estimates of other Total Annual Cost Burden to Respondents or Recordkeepers/Capital Costs

There will be no capital costs or costs associated with operations and maintenance to respondents as all data are reported through a web-based enterprise system owned by and maintained at HRSA.

14. Annualized Cost to Federal Government

The systems used to collect information in the form of progress reports and the PRGCA are maintained by HRSA. It is estimated that the amount of staff time needed for the review and approval of progress reports and PRGCA submitted on an annual basis is equivalent to 2 FTEs at the GS-13, Step 1 (Washington-Baltimore-Arlington, DC-MD-VA-WV-PA locality) level in 2023—for a total of $224,030. Wage has been multiplied by 1.5 to account for overhead costs. Collectively, the estimated annualized cost to the government in staff time is estimated to be $336,045.

15. Explanation for Program Changes or Adjustments

There are currently 5,402 total burden hours approved by OMB for this activity. This request is for approval of roughly 5,929 burden hours, an increase of 527 hours. The increase in burden is due to the growth in the number of awardees. Without the additional awardees, the burden hours would have decreased as a result of: 1) implementation of an Excel upload option for individual-level data reporting for all awardees, 2) decreased usage of time-consuming forms such as the Grant Purpose and LRDV forms, and 3) NCHWA scientists’ responsiveness to feedback on the data collection process and ongoing efforts to improve efficiency. Specifically, providing all programs with the ability to enter and upload individual-level data via an Excel spreadsheet allows awardees to work offline and complete data entry more quickly. It also allows for copy/paste functionality from external sources. Awardees have provided very positive feedback about the Excel upload feature and have stated that it greatly reduces the time needed to complete their reporting requirements. Additionally, BHW split profession/discipline questions into two parts. This reduced the number of selection options for this lengthy question. Awardees reported that it was difficult and time consuming to find the option they were looking for. In response, BHW created an intermediary question for “type” of profession (i.e., dental, medical, etc.) to make it easier for respondents to then locate the more specific discipline within the existing profession question. These professions and disciplines were also standardized across all forms that collect this information. BHW has also maintained the significant burden reduction of over 1,700 hours in 2016 by continuing to use the annual reporting strategy.

Additional proposed changes and additions do not increase burden for awardees. BHW proposes to standardize partnership questions across EXP and PC forms to better understand the purposes of the partnerships and how partnerships affect trainee outcomes. New questions ask for the primary and secondary purposes of the partnerships. Using the same options and the same questions across forms simplifies reporting for awardees. BHW plans to carry the data forward each year to reduce reporting burden. The data for partnerships only needs to be updated annually if there is a change. The proposed employment-related questions standardize the employment location, type of employment, and hiring organization questions across the INDGEN and the INDGEN-PY forms. The questions ask awardees to respond ‘Yes’ or ‘No’ to whether their organization or a partner organization hired the individual and to enter the employment zip code and the type of employment location setting for trainees. This data will be collected at two time points – at graduation and one year after program completion. Having the same questions with standardized options on both forms will improve BHW’s ability to measure outcomes and will simplify reporting for the grantees. For the location data, once the ZIP Code is entered, the system will automatically generate the city and state, reducing the burden on the grantee. The additional gender questions BHW proposes adding to LR2 comply with the Executive Order 14075 and improves reporting accuracy for gender. Awardees have long requested additional gender options and this update addresses this request. The INDGEN question on supplemental funding is specific to one grant program and will prevent awardees from having to complete multiple duplicative APRs. The apprenticeship questions are intended to make it possible to determine outcomes specific to the apprenticeship component of grant programs that offer apprenticeships. These questions align with the data collected by the Department of Labor’s on apprenticeship programs. This alignment will ensure consistency in data collection efforts and simplifying reporting for awardees. The apprenticeship questions will allow BHW to understand employment outcomes and the skills and competencies acquired during apprenticeships.

There are approximately 50 different forms being used for reporting. The measures in each of these forms are provided in Appendix C. These forms were redesigned in 2012, and they have provided BHW with exceptional results. More importantly, they have improved the quality and accuracy of data reported. The EHB system continues to provide user-friendly templates on certain forms as well as pre-populated data fields on forms that do not utilize templates. The web-based reporting system features reduce the need for manual data entry thus reducing burden. The system pre-populates fields with previously entered data thus reducing data re-entry by the user; automate the calculation of total counts; and allow awardees the ability to enter data into spreadsheets that are available within the web-based reporting system. Several forms have the option to update previous information reported—reducing the need to re-report information which does not vary during the life of a specific grant. For example, demographic information about individuals receiving direct financial support (e.g., stipends, loans, or scholarships) will only have to be reported once. Using awardee-developed unique identifiers or NPI numbers, each awardee will only be required to update specific fields—such as financial award amounts, attrition status, graduation status, and one-year follow-up.

In 2020, BHW launched HTML versions of PRGCA manuals. Moving from a PDF to an HTML version of the PRGCA manual provides awardees a more dynamic experience. Manuals can be updated in real-time, and awardees can access them directly from the EHB. By linking the manuals within EHB, awardees automatically get directed to the section of the manual that relates to the form they are working on. These enhancements reduce time spent searching for answers to common questions. Manuals can also be updated in real-time to address confusion during the reporting cycle. The PRGCA manuals contain specific instructions for awardees of each program. As each manual is tailored to the specific program reporting needs, awardees only see forms and items they must complete and are not confused by forms or fields that are not relevant to their grant program. The awardees have provided very positive feedback on the user-friendliness and the improved efficiency of the manuals when they are reporting performance data. The manuals as well as the applicable performance measures are available online at http://bhw.hrsa.gov/grants/reporting/index.html. These manuals serve to increase reliability and accuracy of the performance data and serve an important role in BHW’s performance management strategy.

16. Plans for Tabulation, Publication, and Project Time Schedule

Phase I: Mid-Year Progress Reports - Data collected in the form of progress reports will serve as the official record of communication between government project officers and awardees and will be used to respond to awardee-specific concerns and technical assistance needs.

Phase II: End-of-Year Annual Performance Reports - Data collected in the form of PRGCA serve a number of important purposes including strengthening program performance; responding to Federal reporting requirements (e.g., GPRAMA); responding to Congressional inquiries. Since programs are publicly-funded, data collected through the PRGCA may be showcased in peer-reviewed articles, conferences, reports, and/or dashboards published through and/or sponsored by HRSA. The process for cleaning, analyzing, and reporting data will consist of the following steps⁵:

Step 1: Data completeness and accuracy. BHW utilizes a multi-level approach to ensure that data/information used for performance measures is complete. Awardees enter required data according to an established data reporting process which, for most of its workforce programs, includes reporting through HRSA’s grant system, EHB. During data entry in EHB, validation checks on data ensure report completeness and consistency across related measures. The awardee is notified during report completion of any errors that must be resolved prior to submission. Following report submission, government project officers (POs) review the performance report for accuracy and completeness. If the information is incomplete, the PO will request changes from the awardee and the report is sent back to the awardee. The awardee will then resubmit a report after addressing the PO’s concerns. Lastly, the data are reviewed again for completeness and accuracy by scientists in NCHWA who maintain the performance measures and are knowledgeable about program requirements. Deadlines for reporting are consistent with the Uniform Guidance (45 CFR 75) and shown in Table 3.

Data collected through performance reports serve a number of critical functions such as informing program management decisions, monitoring the types of activities implemented by awardees, and enhancing HRSA’s understanding about the diversity and distribution of the individuals receiving direct support from HRSA programs. In addition, performance reports are essential for:

• responding to federal reporting requirements;

• understanding emerging issues in the health professions;

• ensuring compliance with statutory requirements, applicable regulations, and terms and conditions of award;

• identifying potential promising or best practices;

• strengthening overall program performance; and

• responding to congressional and public inquiries regarding outcomes associated with health professions training and loan programs.

The programs receive and review raw data from awardees, aggregate these data for HRSA-level reporting, have ongoing oversight of reporting entities and have direct contact with them to resolve potential data problems. For instance, if an awardee falls short of its performance targets, the PO will work to understand why and may work with the awardee to develop a corrective action plan. These data are also reviewed by NCHWA scientists who use their data and evaluation expertise to assess how the data relate to national trends. In 2021, NCHWA launched Grantee Scorecards that allow awardees, POs, NCHWA scientists, and other BHW staff to see program performance as compared to program and national targets. The Scorecard provides a visual snapshot of performance that can be used for program management and evaluative purposes. In addition, measures are entered into the HHS Data Analytics System (DAS). DAS is used to produce information for the HHS Annual Performance Report.

Step 2: Analysis⁶ & Reporting. The analysis of all PRGCA-related data will be conducted by doctoral-level scientists in NCHWA according to the following priority-based schedule:

1. Priority I. Data that is essential for performance management and budgeting will be analyzed with the highest priority. Results from these analyses will be provided to government project officers and BHW leadership in the form of briefs and/or reports within 30 days of NCHWA scientists completing the data cleaning process.

2. Priority II. Data that can be used to respond to inquiries from Congress, stakeholders, and/or the public will be analyzed and reported in accordance with the urgency of the request (usually 1 to 3 days).

3. Priority III. Data that can enhance the agency's understanding of emerging trends in the health professions will be analyzed and provided to BHW leadership in the form of dashboards, briefs, or presentations within 60 days of NCHWA scientists completing the data cleaning process.

4. Priority IV. Data that can be used to inform the development of articles or conferences will be analyzed and abstracts of findings will be provided to the requesting staff usually 2-4 weeks from the day of the request.

Aggregate data from this information collection will be published by HRSA on https://data.hrsa.gov/topics/health-workforce/training-programs.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

No exemption is requested. Respondents will see the OMB number and expiration date on each table that exists in the EHB system.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

This information collection fully complies with the guidelines set forth in 5 CFR 1320.9. There are no exceptions to the certification statement.

Attachments

• APPENDIX A: BHW-funded Health Professions Training and Loan Programs by Program Type

• APPENDIX B: Measures to Be Collected from Funded Awardees through Progress Reports

• APPENDIX C: Measures to Be Collected from Funded Awardees through the PRGCA

APPENDIX A

BHW-funded Health Professions Training and Loan Programs by Program Type

Type	Activity Code	Program
Direct Financial Support	A22	Nurse Anesthetist Traineeship
	D87	Dental Faculty Loan Repayment
	E01	Nurse Faculty Loan Program
	H56	National Health Service Corps-State Loan Repayment Program
	K01	Geriatrics Academic Career Awards
	K02	Primary Care Medicine and Dentistry-Clinician Educator Career Development Awards
	T08	Scholarships for Disadvantaged Students
	T23	Children's Hospitals Graduate Medical Education
	T25	Addiction Medicine Fellowship
	T32	National Research Service Award in Primary Care
	T52	Public Health Scholarship Program
	T91	Teaching Health Center Graduate Medical Education
	T96	Advanced Nursing Education (ANE)-Sexual Assault Nurse Examiners
	T9C	Teaching Health Center Graduate Medical Education Program
Infrastructure Programs	P13	Rural Residency Planning and Development Program
	T9B	Teaching Health Center Planning and Development
	U3M	Promoting Resilience and Mental Health Among the Health Professional Workforce
	U3N	Health and Public Safety Workforce Resiliency Training Program
	U4E	Nurse Education, Practice, Quality, and Retention (NEPQR)-Simulation Education Training
	U77	Area Health Education Centers
	UK1	NEPQR-Mobile Health Training Program
Multipurpose or Hybrid Programs	D18	Health Careers Opportunity Program
	D19	Nurse Workforce Diversity
	D33	Preventive Medicine Residency
	D34	Centers of Excellence
	D40	Graduate Psychology Education
	D57	Physician Assistant Training in Primary Care
	D58	Residency Training in Primary Care
	D85	Predoctoral Training in General Dentistry, Pediatric Dentistry, and Dental Public Health and Dental Hygiene
	D88	Postdoctoral Training in General, Pediatric, and Public Health Dentistry
	M01	Behavioral Health Workforce Education and Training
	MC1	Behavioral Health Workforce Education and Training-American Rescue Plan
	T0B	Primary Care Training Enhancement (PCTE)-Integrating Behavioral Health in Primary Care
	T12	State Oral Health Workforce Program
	T13	PCTE-Training Primary Care Champions
	T1Q	NEPQR-Clinical Faculty and Preceptor Academies Program
	T26	Opioid-Impacted Family Support Program
	T29	Community Health Worker Training Program
	T34	PCTE-Community Prevention and Maternal Health
	T59	ANE-Nurse Practitioner Residency and Fellowship Program**
	T5C	PCTE-Language and Disability Access**
	T62	Integrated Substance Use Disorder Training Program
	T68	Maternity Care Nursing Workforce Expansion Program**
	T94	Advanced Nursing Education Workforce
	T99	Value-Based Medical Student Education Training
	T9D	PCTE-Physician Assistant Rural Training Program
	TA2	PCTE-Residency Training in Mental and Behavioral Health**
	U1Q	Geriatrics Workforce Enhancement Program
	U68	State Primary Care Offices
	UB6	Public Health Training Centers
	UD7	NEPQR-Pathway to Registered Nurse Program**
	US6	NEPQR-Registered Nurse Training Program

Note: ** identifies the new programs for Academic Year 2023-2024. These programs are included in the burden estimates based on their expected reporting requirements.

APPENDIX B

Progress Report

Public Burden Statement: The BHW Performance Report for Grants and Cooperative Agreements (PRGCA) is an annual performance and progress report required from each health professions and nursing education grantee that has an approved, funded project with a project period of one year or more. The report is required to determine the extent to which objectives of the project have been met so that a decision regarding continuation funding can be made. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this information collection is 0915-0061 and it is valid until 03/31/2025. This information collection is required to obtain or retain a benefit (Government Performance and Results Act (GPRA) of 1993 and the GPRA Modernization Act of 2010). The information will be kept private to the extent permitted by law (see 42 USC 292 et seq). Public reporting burden for this collection of information is estimated to average 0.5 hours per response to the annual performance report, including the time for reviewing instructions, searching existing data sources, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to HRSA Reports Clearance Officer, 5600 Fishers Lane, Room 14N39, Rockville, Maryland, 20857 or paperwork@hrsa.gov. 5600 Fishers Lane, Room 10-29, Rockville, Maryland, 20857.

FORM NAME: PROGRESS REPORT

TYPE OF RESPONDENT: Awardees of Infrastructure Programs

Awardees of Direct Financial Support Programs

Awardees of Multipurpose or Hybrid Programs

SECTION I. PROJECT OBJECTIVES AND ACCOMPLISHMENTS⁷

Objective A

Description of Objective

Accomplishments

Objective B

Description of Objective

Accomplishments

Objective C

Description of Objective

Accomplishments

SECTION II. BARRIERS & RESOLUTIONS⁸

Barrier A

Description

Activities Taken to Resolve

Barrier B

Description

Activities Taken to Resolve

SECTION III. TECHNICAL ASSISTANCE NEEDS

Please identify any technical assistance needs that will assist your organization in meeting project objectives and/or improve performance.

APPENDIX C

PERFORMANCE REPORT FOR GRANTS AND COOPERATIVE AGREEMENTS

See attached documents. Attached are the performance measures and mapping documents. The mapping documents indicate which forms and items on the form that each awardee completes.

1 Pub. L. No. 103-62, 107 Stat. 285 (Aug. 3, 1993).

2 Pub. L. No. 111-352, 124 Stat. 3866 (Jan. 4, 2011).

3 Pub. L. No. 115-435, 132 Stat, 5529 (Jan. 14, 2019).

4 See Appendix A for a complete listing of BHW funded programs by category.

5 Steps apply to each reporting period (FY 2016 and beyond). Please see Table 3 for an overview of beginning and ending periods of reporting.

6 The analyses of PRGCA data have historically been primarily descriptive in nature (e.g., frequencies, percentages, ratios).

7 Note: awardees will have the ability to list up to 9 objectives and related accomplishments.

8 Note: awardees will have the ability to list up to 9 barriers and related solutions.

Page 29 of 29