Survey on Evaluation Technical Assistance Received by State Peer Cohorts

Request for Approval under the “Quick Turnaround Surveys on Workforce Investment Act Implementation”

(OMB Control Number: 1205-0436)

TITLE OF INFORMATION COLLECTION:

Survey on Evaluation Technical Assistance Received by State Peer Cohorts

PURPOSE:

This electronic survey intends to assess, and identify ways to improve, the effectiveness of an annual series of evaluation research-related training resources and coaching activities. made available for State Workforce Agency managers and staff (SWAs) and other state partner agencies. These resources and activities are among of several forms of the technical assistance (TA) provided to help SWAs meet requirements to conduct evaluations under the Workforce Innovation and Opportunity Act (WIOA) and the regulations at 20 CFR 682.220 in coordination with other state partner programs. The survey will be conducted by a Mathematica under a contract with the Employment and Training Administration (ETA) in the U.S. Department of Labor.

Topics covered: Evaluation technical assistance provided through the EvalPLC leads to knowledge and skills growth through the assessments and action planning, experiences during virtual events and coaching, and tools and resources used to develop evaluation capacity. Responses to the surveys will inform ETA about how to improve its technical assistance approach for evaluation capacity through the EvalPLC.

Key areas of interest in the eventual suggestive findings will be used to:

• Assess and understand the effectiveness of the TA provided in the EvalPLC activities;

• Further identify strategies to improve the EvalPLC activities, interest areas to expand evaluation research capacity, and coaching provided; and

• Improve the activities conducted with the EvalPLC TA for future state team cohorts and other evaluation capacity resources and products of interest, such as related webinars or toolkits.

Data collection methods and the likely response rate (for surveys): The survey will be conducted online, via the survey platform QuestionPro, in early 2025. The survey will be released in two waves: the first wave to the five earlier cohorts and the second wave to the current, 2024-2025 cohort. Respondents will be contacted primarily by email. We will also reach out by phone or mail to nonrespondents. Respondents who are no longer employed at the SAWs or partner agencies will be removed from further outreach. We anticipate fielding the survey for ten weeks for the earlier cohort and eight weeks for the current, 2024-2025 cohort. Given the online format of the survey, ETA anticipates at least an 80 percent response rate.

Brief review of analytical methods and the statistical methodology: The universe has been preferentially selected for this survey from all the EvalPLC state teams; therefore, it is not subject to any sampling. The lack of national data about these participants precludes any nationally representative determinations, a non-response bias analysis, and the need for statistical adjustments. Since there is no sampling and no statistical analysis to be undertaken, no statistical review has been conducted in advance. Information from the data collected in the survey will be treated as strictly suggestive and no statistical inferences will be made.

Privacy protections for respondents and data security measures: This survey is conducted in accordance with all relevant regulations and requirements, including the Privacy Act of 1974 (5 U.S.C. § 552a). Although the names of each EvalPLC state team member will be used by the contractor to send out the instruments and to determine if surveys have been completed, no information collected and analyzed will be linked to any individual respondents. Further, names of respondents will not be included in published reports nor otherwise be revealed to anyone not directly involved in the information collection (i.e., contractor personnel). Respondents will be advised that their responses will be kept private, and that the information they provide will be combined with that from other individuals and presented in a report which will not identify any nor permit identification of any individual respondents. The survey and outreach materials will include the statement: “All information you provide will be kept private to the extent permitted by law. You will not be required to provide personal identifiers such as your name or address and your personal information will not be disclosed or identified in any report.”

The intended use of the resulting information and data: The data collected under this survey will be analyzed and results provided to ETA in a report, which will be used to understand the experiences of the EvalPLC state team members and to identify possible changes in the EvalPLC to make it more effective.

Current Inventory of Surveys and Site Visits under OMB control number 1205-0436: There are currently no active surveys being conducted under the OMB # 1205-0436 (expiring in November 2026) that can include up to 7,000 responses approved for this collection. The total burden of 60 hours for this survey will still allow for multiple other surveys under the overall burden level of 7,000 hours over three years.

DESCRIPTION OF RESPONDENTS:

This survey will be sent to all 240 individuals who participated in EvalPLC state teams over six cycles, including the 2024-2025 sessions and five earlier cohorts (whose training occurred from 2019 through 2024). Survey participants include mangers and staff in different roles in the states, including those involved in research as well as in data collection and analysis related existing data systems used for program performance and labor market information.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [X] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software [ ] Small Discussion Group

[ ] Focus Group [ ] Other: ______________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: __Gloria Salas Kos______________________

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [] Yes [X] No

2. If Yes, is the information that will be collected included in records that are subject to the Privacy Act of 1974? [ ] Yes [ ] No

3. If Applicable, has a System or Records Notice been published? [] Yes [] No

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [] Yes [X] No

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Burden
State workforce agency and partner program representatives	240	15 minutes	60 hours
Totals	240	15 minutes	60 hours

FEDERAL COST: The estimated annual cost to the Federal government is $67,020.00__

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [X] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

We have a customer list that defines the universe of potential respondents, and the survey will be distributed to the complete list. Sampling will not be used.

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[X] Web-based or other forms of Social Media

Percentage of Respondents Reporting Electronically:

[X] Telephone

[ ] In-person

[ ] Mail

[ ] Other, Explain

2. Will interviewers or facilitators be used? [ ] Yes [X] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

Instructions for completing Request for Approval under the a Generic Clearance

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the subject of the request. (e.g. Comment card for soliciting feedback on XXXXX)

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used. If this is part of a larger study or effort, please include this in your explanation.

DESCRIPTION OF RESPONDENTS: Provide a brief description of the targeted group or groups for this collection of information. These groups must have experience with the program.

TYPE OF COLLECTION: Check one box. If you are requesting approval of other instruments under the generic, you must complete a form for each instrument.

CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the collection will be returned as improperly submitted or it will be disapproved.

Personally Identifiable Information: Provide answers to the questions.

Gifts or Payments: If you answer yes to the question, please describe the incentive and provide a justification for the amount.

BURDEN HOURS:

Category of Respondents: Identify who you expect the respondents to be in terms of the following categories: (1) Individuals or Households;(2) Private Sector; (3) State, local, or tribal governments; or (4) Federal Government. Only one type of respondent can be selected.

No. of Respondents: Provide an estimate of the Number of respondents.

Participation Time: Provide an estimate of the amount of time required for a respondent to participate (e.g. fill out a survey or participate in a focus group)

Burden: Provide the Annual burden hours.

FEDERAL COST: Provide an estimate of the annual cost to the Federal government.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents. Please provide a description of how you plan to identify your potential group of respondents and how you will select them. If the answer is yes, to the first question, you may provide the sampling plan in an attachment.

Administration of the Instrument: Identify how the information will be collected. More than one box may be checked. Indicate whether there will be interviewers (e.g. for surveys) or facilitators (e.g., for focus groups) used.

Please make sure that all instruments, instructions, and scripts are submitted with the request.

7