0524-0050 Supp. Stmt. B

National Institute of Food and Agriculture

U.S. Department of Agriculture

OMB No. 0524-0050

Veterinary Medicine Loan Repayment Program

SUBJECT: Supporting Statement B. Statistical methods for surveys associated with the Veterinary Medicine Loan Repayment Program (VMLRP).

B. STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate during the last collection.

1. State Animal Health Official, Applicant and Awardee Surveys. Each State Animal Health Officials (SAHOs) will receive a Survey of Animal Health Officials Completing the Veterinarian Shortage Nomination Forms. There are approximately 60 SAHO’s. Each VMLRP award cohort consists of approximately 150 applicants and 80 participants. Results will not be used to make statements representative of all VMLRP participants and are not intended to be generalizable beyond the respondent group.

2. Post-Award Survey. All participants will receive a post-award termination survey at one, three, and five years, post-agreement completion. Assuming an average of 80 participants each year, 76 participants need to respond to have a confidence level of 95% and 5% margin of error in the calculated retention rate for a particular cohort. The program is interested in the retention rate of veterinarians in shortage situations for the program as a whole, versus for each cohort, at one, three and five years, post-agreement; therefore, cohorts will be merged based on the number of years post-agreement, as there is no reason to assume the factors surrounding retention would be any different among cohorts.

Response rate will be participant driven. The National Health Services Corps (NHSC), a program for physicians that is similar to VMLRP, surveys their loan repayment participants, one-year post-agreement and achieves a 58.9% to 75.2% response rate. For VMLRP, we estimate the response rate will be at least 65%. Thus, with the current number of participants, we do not anticipate having a large enough sample of respondents for statistical analysis until we have surveyed four cohorts, at each post-agreement time point. Until these numbers are reached, retention rates will not be considered generalizable to the program and all analyses will be restricted to the respondent group.

2. INFORMATION COLLECTION PROCEDURES

Describe the procedures for the collection of information including:

1. Statistical methodology for stratification and sample selection;

2. Estimation procedure;

3. Degree of accuracy needed for the purpose described in the justification;

4. Unusual problems requiring specialized sampling procedures; and,

5. Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

No estimation or stratification is used because NIFA will survey 100% of the VMLRP applicant and participant population. Applicant and participant contact information will be obtained from the VMLRP administrative database. This database is continually updated as new participants join the program and as status or contact information changes for participants. The VMLRP also uses information from the database to generate all reports about the program. Quarterly communications via email (and phone if needed) occur between VMLRP staff and participants throughout the program and continues at least one year after completion of an agreement for tax purposes. Upon completion of a VMLRP agreement the information is retained.

1. Applicant and Awardee Feedback Surveys. Applicant and participant contact information will be obtained from the VMLRP database. Applicants and participants will be e-mailed a link to a web-based survey instrument or, if a web-based survey instrument cannot be used, the survey will be attached to an email as a pdf-fillable form. Applicants will receive the survey after the deadline of application is complete. Participants will also receive the survey after they complete the first year of their agreement.

We are interested in qualitative information about the program. All analyses will involve frequencies and text analysis.

2. Post-Award Survey. Participant contact information will be obtained from the VMLRP database. The VMLRP database is not updated after one-year post-agreement. Therefore, as time goes on it is possible VMLRP will lose contact with some past participants due to lack of updated contact information. Attempts will be made to obtain updated contact information (see Section 3b below). Non-response due to lack of updated contact information was assumed when estimating the anticipated response rate of 65%.

At one, three, and five years, post-agreement, participants will be e-mailed a link to a web-based survey instrument or, if a web-based survey instrument cannot be used, the survey will be attached to an email as a PDF-fillable form.

Data from responses received in electronic format will be directly loaded into a database. Responses received in hard-copy format will be keyed, edited, entered in the database manually, and reviewed by another person for accuracy.

Retention in our analyses will be defined as a proportion of participants retained in shortage situations at one, three, and five years, post-agreement. Reasons for retention or lack of retention will be described descriptively with no generalizations.

3. METHODS TO MAXIMIZE RESPONSE RATES

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

1. SAHO, Applicant and Awardee Feedback Surveys. Information collected for these surveys will not yield generalizable quantitative findings; it can provide useful customer input, but it will not yield data about customer opinions that can be generalized.

2. Post-Award Survey. All surveys will be sent electronically. To accommodate those not wanting to respond via the Internet, the email invitation will offer the option to request a PDF-fillable form (when a web-based survey instrument is used). Additionally, non-respondents will be sent a reminder along with the link or copy of the survey two, four and six weeks after the initial email inviting them to take the survey.

Rejected emails and other indications of bad email addresses will be followed up by attempting to contact the participant by phone at the number on file and/or at the last place of known employment. If all contact information on file appears to be outdated the state veterinary board in which the participant was last known to be practicing will be contacted for current contact information.

Surveys are sent to a population in which the program already has relevant demographic information. Non-respondents’ demographics will be compared to those responding to the survey and assessed for any significant differences. Any significant differences found will be adjusted for in the overall analysis.

3. TESTS OF PROCEDURES

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

All survey questions were developed based on the needs of the program and after reviewing survey questions from other loan repayment programs assessing retention. Pretesting was done with internal staff, external working group and four current or past VMLRP participants.

3. STATISTICAL CONSULTANTS

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

NIFA did not consult with any outside individual on the statistical aspects of the design of its survey. NIFA employees who will collect and/or analyze the information are:

Kathe Bjork

816-591-7415

Marline Azevedo

816-401-7790

3