Factors That Influence Effectiveness
ICR Part B
Information Collection Request Supporting Statement: Part B
Factors That Influence Effectiveness of Hazard Anticipation and Attention Maintenance Training
OMB Control No. 2127-New
Abstract:1
The National Highway Traffic Safety Administration (NHTSA) is seeking approval for a one-time voluntary information collection from 168 participants ages 18 and 19 who do not yet have driver’s licenses for a research study on novice driver training. Specifically, this collection involves developing and testing a novice driver training program on a smartphone-like platform and determining whether the effectiveness of the training differs for participants of different sexes, socio-economic status (SES) strata, and trait levels of sensation seeking and aggressiveness.
The information collection will proceed in two phases. First, the research team expects to provide screening questionnaires to 1,002 potential participants to determine their eligibility for the study; screening questionnaires will be provided to those who respond to recruitment information about the study posted on social media platforms or provided to contacts in local communities (e.g., teen centers, community college faculty in the area, high school principals, local driving schools) and to those who participated in past studies at the research center and agreed to be contacted about future opportunities. To be eligible, participants must be 18 or 19 years old, must not have an unrestricted driver’s license or an intermediate/provisional license that allows driving independently, and must be interested in obtaining an unrestricted or intermediate/provisional license in the next 12 months. Participants may have a learner’s permit. Completing the screening questionnaire is estimated to have a burden of 5 minutes per respondent (an annual burden of 28 hours for 334 annual respondents, averaged over the three-year approval period). Second, of the estimated 1,002 potential participants who are provided a screening questionnaire, an estimated 180 respondents are expected to complete the screening questionnaire and be eligible, interested, and willing to travel to the research center to undergo the informed consent process, with an estimated burden of 70 minutes per respondent, including travel time (an annual burden of 70 hours for 60 annual respondents).
Finally, of the estimated 180 respondents who undergo the informed consent process, an estimated 168 respondents are expected to consent and enroll in the study. These participants will complete the enrollment process and a pre-study questionnaire, with an estimated burden of 10 minutes per respondent (an annual burden of 9 hours for 56 annual respondents). Then, participants will complete the novice driver training protocol, which involves a pre-training test, the hazard anticipation and attention maintenance training program (or a placebo training program), a post-training test (all administered on a computer), a rest break, a test on a computerized driving simulator, and a post-study questionnaire. The training protocol has an estimated burden of 230 minutes per respondent (an annual burden of 215 hours for 56 annual respondents). The total annual burden for this information collection is 322 hours and $13,069.
Prior to conducting the study, the research team will obtain review and approval of this data collection from an Institutional Review Board (IRB) that meets all Federal requirements in 45 CFR 46, is registered with the Office for Human Research Protections, and has a Federalwide Assurance. NHTSA will use the results of this study to produce a technical report containing summary descriptive and inferential statistics. No identifying information or individual responses will be reported. The technical report will be shared with State highway safety offices, local governments, policymakers, researchers, educators, advocates, and others who may wish to use the data from this survey to support their work on novice and teen driver safety.
B. JUSTIFICATION
The respondent universe is comprised of all residents of the Boston metropolitan area (one potential study site location) who are either 18 or 19 years old and do not currently hold a provisional/intermediate or unrestricted driver’s license.2 Based on the 2022 American Community Survey (ACS) 1-year estimates, there were 144,632 people ages 18 or 19 in the Boston metropolitan area.3 Additionally, per the Federal Highway Administration,4 there were 113,583 licensed drivers ages 18 or 19 in the State of Massachusetts in 2022. (Note that FHWA’s data is only available at the State level.) Using the ACS 1-year estimate for the population of 18- and 19-year-olds in Massachusetts in 2022 (207,030),5 we estimate that 54% of people ages 18 or 19 in Massachusetts had a driver’s license in 2022, while 46% did not have a license. Applying this same percentage to the Boston metropolitan area, we estimate that there are 66,531 people ages 18 and 19 without a driver’s license. Thus, the size of the target population is approximately 66,000 people.
Respondent Sampling
We estimate that 1,002 potential participants will respond to recruitment efforts (e.g., those who respond to recruitment information posted on social media platforms or provided to existing contacts in local communities) and will be sent a screening questionnaire to complete electronically (see ICR Part B, Appendix 1). Similar to another NHTSA study recently reviewed by OMB,6 we estimate that approximately 18% of these potential participants (180 participants) will complete the screening questionnaire and will be eligible, interested, and willing to travel to the research center to undergo the informed consent procedure. Then, an estimated 168 participants (approximately 94% of those who undergo the informed consent process) are expected to consent and enroll in the study.
This is an experimental study that will compare performance on a computerized driving simulator for a group of participants that receive hazard anticipation and attention maintenance training and a group of participants that receive placebo training. Participants in this study are not expected nor intended to be a representative sample of drivers ages 18 or 19 without driver’s licenses in the Boston metropolitan area. However, care will be taken to recruit participants from a wide cross-section of the population in the study area in an attempt to offer people of all demographic groups (e.g., race and ethnicity, sex, socioeconomic status) the opportunity to volunteer to participate. The research team will post the opportunity on social media platforms likely to be seen by eligible participants and provide it to existing contacts (e.g., managers at neighborhood development community centers, teen centers, community college faculty in the area, high school principals, local driving schools) in local communities. The research team will also contact potential participants via e-mail or phone who indicated a prior interest in similar past studies if they agreed to be contacted about future opportunities.
Participants will be randomly assigned to complete the hazard anticipation and attention maintenance training (the treatment group) or a placebo training (the control group). To minimize baseline differences between the two groups (treatment and placebo) and ensure sufficient power to detect differences in training effectiveness by sex, quota sampling will be used for sex: half of participants in each training group will report being of male sex. Additionally, for the same reasons, quota sampling will be used for socioeconomic status (SES). SES will be assessed by the zip code of a participant’s residence when they were 17, since some participants (ages 18 and 19) may be living at a college or university with a zip code not reflective of the participant’s SES. For quota sampling, among the male participants in each group, half will have lived in areas where the zip code indicates the poverty rate is at or above 20%, and the other half will have lived in areas where the zip code indicates that the poverty rate is below 20%. The same procedure will be used for females in each group. Such a division is made because it is known that crash rates are higher for young novice drivers who are male7 and for young novice drivers who live in areas where the poverty rate is greater than or equal to 20%.8 Because the research team will target some recruitment efforts (i.e., through existing contacts at local teen centers, community college faculty in the area, high school principals, local driving schools) at neighborhoods in which the poverty rate is at or above 20%, we estimate that responses to recruitment efforts by 1,002 potential participants will be sufficient to yield a final sample of 84 participants in each SES strata (N = 168 total).
Before the experiment begins, the research team will obtain informed consent. The consent form will be approved by an IRB that meets all Federal requirements in 45 CFR 46, is registered with the Office for Human Research Protections, and has a Federalwide Assurance. A total of 168 participants will be enrolled (Table 1). Half of the eligible participants will be randomly assigned to the treatment training (the novice driver training program), the other half to placebo training, subject to the constraint that in the treatment and placebo groups, half of participants will be males and half females. Additionally, among the males and females in each group, half will have lived (at age 17) in areas with zip codes that have poverty rates of 20% or greater (low SES) and half with poverty rates of less than 20% (mid+ SES).
Table 1. Expected participant distribution across training group, SES, and sex
|
Hazard Anticipation & Attention Maintenance Training |
Placebo Training |
||
|
Low SES |
Mid+ SES |
Low SES |
Mid+ SES |
Male |
21 |
21 |
21 |
21 |
Female |
21 |
21 |
21 |
21 |
After completing the informed consent and enrollment processes, all participants will complete a pre-study questionnaire to collect information about a participant’s propensity for experiencing motion sickness in the computerized driving simulator. Then, participants will complete a pre-training test of hazard anticipation and attention maintenance skills, administered on a computer. Measures collected during the pre-training test include the location where the participant clicks on the screen (an indication of whether the participant anticipates a latent hazard or not9) and how long the participant spends glancing at the secondary task each time the participant decides to display it on the screen (a proxy for the duration of off-road glances10). No feedback is given during the pre-training test.
Next, participants will complete either the hazard anticipation and attention maintenance training or a placebo training. The hazard anticipation training teaches participants to identify the location of potential hazards; information is presented about various scenarios in which potential hazards exist, and participants are also given the opportunity to practice what they’ve learned. The attention maintenance training informs participants of the risks associated with long glances away from the roadway, and participants learn and practice using repeated brief glances to complete tasks while driving. During the hazard anticipation and attention maintenance trainings, participants are given feedback about their mastery of each skill (i.e., correct location of a potential hazard or no secondary task worked on for more than 2 seconds). The placebo training involves teaching participants the rules and regulations in the driver manual for the State in which the study is conducted. After training, all participants will complete another test of hazard anticipation and attention maintenance skills (post-training test) and will receive a 15-minute rest break. The same measures are collected in the post-training test as during the pre-training test, and no feedback is given.
Then, to assess the effectiveness of the training on a platform different than the platform used in the training, all participants will complete a 90-minute drive on a computerized driving simulator. During the drive, each participant’s eye glance behavior will be measured with a head mounted eye tracker placed on the participant while they are navigating scenarios on the driving simulator. Audio of any verbalizations a participant makes during the drive will also be recorded. Vehicle behaviors will be collected automatically by the simulator software and include velocity, brake pedal pressure, and location on the roadway, among others. After the simulator evaluation, all participants will complete a post-study questionnaire that includes questions assessing: the potential for motion-related driving simulator sickness, demographics, driving experience; the personality traits of sensation seeking and aggressiveness; and driving behavior. Finally, just before leaving participants will receive final payment in the form of a $165 Amazon gift card.
The primary dependent measures for assessing the effectiveness of the training will be collected with the eye tracker during the drive on the computerized driving simulator. For each participant, there are a total of 36 binary measures of hazard anticipation (i.e., either the participant does or does not glance towards the area of the roadway that provides information about a hazard further downstream or immediately ahead). Additionally, for each participant, there are 18 proportions that reflect the proportion of glances away from the forward roadway greater than 2 seconds during the performance of each of 18 secondary tasks (attention maintenance).
A novice driver training program that contains hazard anticipation and attention maintenance components will be evaluated in a rigorous, randomized controlled trial using the above potential respondent universe. The data collection will be used to learn: (a) whether there is an effect of each of the two training components (hazard anticipation and attention maintenance) on behaviors measured on a driving simulator that are known to be linked to crash risk; and (b) whether the effectiveness of the training program differs between participants by SES (high vs. mid+), sex (male vs. female), or trait levels of sensation seeking and aggressiveness. Scores on the measures of sensation seeking11 and aggressiveness12 will be used to either categorize participants into higher and lower levels on each or, if the two are highly correlated, a single dimension of combined sensation seeking/aggressiveness (see ICR Part B, Appendices 3 & 4 for more detail on these two surveys). Importantly, a prior study found that drivers ages 18 to 24 with lower scores on these same measures of sensation seeking and aggressiveness (in addition to two other measures) were more likely to anticipate hazards and maintain attention after training than drivers in a control group, whereas drivers with higher levels of sensation seeking and aggressiveness were no more likely to anticipate hazards or maintain attention after training than drivers in a control group.13
The primary independent variable is the training received (treatment or placebo). The dependent variables, which capture the effectiveness of the training program on behavior in a computerized driving simulator, include eye glance behavior for both hazard anticipation (coded as a binary variable: either the driver glanced or did not glance at a location where a clue existed that a latent hazard could materialize) and for attention maintenance (coded as a continuous variable: the proportion of glances at the secondary task longer than 2 seconds). Data will also be automatically collected on vehicle behaviors (e.g., velocity, brake pressure, lane position). However, because prior research has primarily demonstrated the effects of training on novice drivers’ eye glance behavior, these additional dependent variables will only be examined if the patterns of eye glance behaviors are different than expected.
Moderator variables for training effectiveness will include SES, sex, and trait levels of sensation seeking and aggressiveness, as well as age and pre- and post-training test accuracy.
All statistical analyses will be conducted within the R software environment. Analyses will test the following null hypotheses:
H1: There is no main effect of hazard anticipation training on hazard anticipation glance behaviors in the driving simulator;
H2: There are no interactions between SES (H2a), sex (H2b), or level of sensation seeking or aggressiveness (H2c) and the effectiveness of hazard anticipation training on hazard anticipation glance behaviors in the driving simulator;
H3: There is no main effect of attention maintenance training on attention maintenance glance behaviors in the driving simulator;
H4: There are no interactions between SES (H4a), sex (H4b), or level of sensation seeking or aggressiveness (H4c) and the effectiveness of attention maintenance training on attention maintenance glance behaviors in the driving simulator.
The dependent measures will be analyzed using ANOVAs to test the above hypotheses. Specifically, with respect to Hypotheses H1 and H2, we will run four separate ANOVAs: H1 (evaluate effect of training) and H2 (evaluate interaction of training and SES, H2a; training and sex, H2b; and training and sensation seeking and aggressiveness, H2c). Similar analyses will be used for attention maintenance. Prior to analysis, all observations from the driving simulator will be aggregated to the participant level. However, if it did become useful to analyze observations within a participant, we would do so using a logistic, mixed effects regression.
The following data will be collected:
Eye glance, audio, and vehicle data in driving simulator: A head-mounted eye tracker will be used to identify the fixation point throughout a participant’s drive on the driving simulator, using video of the participant’s eyes that has been distorted to remove personally identifying information. The fixation point will then be overlayed on a video of what the driver sees during the drive for later analysis. Manual coders will score the glance data. Audio of any verbalizations a participant makes in the simulator will also be recorded and will be distorted to remove personally identifying information. Vehicle behavior will be automatically recorded by the driving simulator software.
Screening questionnaire: See Appendix 1
Pre-study and post-study questionnaires: See Appendix 2
Data on trait levels of sensation seeking and aggressiveness: See Appendices 3 & 4
This study proposes collecting eye tracking data during a simulated drive on a fixed-based driving simulator. A power analysis was conducted to determine the sample size required to evaluate the hypotheses detailed above, based on assumptions about the patterns of responses expected from participants and the planned analysis approach. All power analyses were performed using G*Power.14
Standard interpretations of Cohen’s f classify values of 0.1, 0.25, and 0.4 as small, medium, and large effect sizes.15 There are four hazard anticipation hypotheses (H1, H2a, H2b, H2c) and four attention maintenance hypotheses (H3, H4a, H4b, H4c) we wish to evaluate. If we want to calculate power for a medium effect size using G*Power, we find that we need a total sample of 128 participants to have an overall probability of a two-tailed Type I error of 0.05 and power of 0.80. In G*Power, the F tests were specified as the test family, a priori was specified as the type of power analysis, and ANOVAs (fixed effects, special, main effects, and interactions) were specified as the statistical tests. There are eight groups [sex (2) X treatment (2) X SES (2), and one moderator, risk seeking level]. Similarly, there are four attention maintenance tests we wish to evaluate. Again, the total sample size we need is 128 to have an overall Type I error of 0.05 and power of 0.80. Given that we expect some participants may decide not to enroll in the study after undergoing the informed consent procedure, we believe 168 total participants are needed.
Participation in the study is voluntary. To maximize participation and minimize the recruitment burden on participants, the research team will post the opportunity on social media platforms likely to be seen by eligible participants and provide it to existing contacts (e.g., managers at neighborhood development community centers, teen centers, community college faculty in the area, high school principals, local driving schools) in local communities. The research team will also contact potential participants via e-mail or phone who indicated a prior interest in similar past studies if they agreed to be contacted about future opportunities.
The research team will also maximize participation and minimize the recruitment burden by offering financial incentives to respondents if they qualify for study participation. As noted above, we will be offering a $165 Amazon gift card to all study participants who complete the study. Experience has shown that this level of compensation is needed to successfully recruit subjects in this type of study which requires, on average, 1 hour of travel and 4 hours and 10 minutes of lab time (see Part A).
Internal pilot testing of screening questionnaires, informed consent forms, training programs (experimental and placebo), pre-training and post-training testing, driving simulator evaluation, demographic questionnaire, simulator sickness questionnaire, surveys, and data collection methods will be undertaken by members of the research team prior to initiating the study to ensure all equipment is working correctly. Any findings from this pilot will be used to make improvements to the data collection instruments and methodology prior to the beginning of actual data collection.
Clark Calabrese, M.A.
Engineering Psychologist
U.S. Department of Transportation
Volpe National Transportation Systems Center
(617) 494-3226
Donald Fisher, Ph.D.
Principal Technical Advisor
U.S. Department of Transportation
Volpe National Transportation Systems Center
(413) 313-4599
Jared Young, M.S.
Engineering Psychologist
U.S. Department of Transportation
Volpe National Transportation Systems Center
(617) 494-2629
Christine Watson, Ph.D.
Project Manager/Research Psychologist
U.S. Department of Transportation
National Highway Traffic Safety Administration
(202) 366-7345
1 The Abstract must include the following information: (1) whether responding to the collection is mandatory, voluntary, or required to obtain or retain a benefit; (2) a description of the entities who must respond; (3) whether the collection is reporting (indicate if a survey), recordkeeping, and/or disclosure; (4) the frequency of the collection (e.g., bi-annual, annual, monthly, weekly, as needed); (5) a description of the information that would be reported, maintained in records, or disclosed; (6) a description of who would receive the information; (7) if the information collection involves approval by an institutional review board, include a statement to that effect; (8) the purpose of the collection; and (9) if a revision, a description of the revision and the change in burden.
2 Participants cannot currently hold or have ever held a provisional/intermediate license or an unrestricted license. A participant may hold a learner’s permit, which allows novices to drive when accompanied by an adult.
6 National Highway Traffic Safety Administration. (2022). Agency information collection: Notice and request for comment: Drivers' knowledge/correct use of new technology features in passenger vehicles. Federal Register, 138 ed., vol. 87. https://www.federalregister.gov/d/2022-15408
7 McCartt, A. & Teoh, E. (2015). Tracking progress in teenage driver crash risk in the United States since the advent of graduated driver licensing programs. Journal of Safety Research (53), 1-9. https://doi.org/10.1016/j.jsr.2015.01.001
8 Males, M. (2009). Poverty as a determinant of young drivers’ fatal crash risks. Journal of Safety Research (40), 443-448. https://doi.org/10.1016/j.jsr.2009.10.001
9 Pollatsek, A., Narayanaan, V., Pradhan, A., & Fisher, D. (2006). Using eye movements to evaluate a PC-based risk awareness perception training program on a driving simulator. Human Factors, 48(3), 255-259. https://doi.org/10.1518/001872006778606787
10 Pradhan, A. K., Divekar, G., Masserang, K., Romoser, M., Zafian, T., Blomberg, R., Thomas, F., Reagan, I., Knodler, M., Pollatsek, A., & Fisher, D. (2011). The effects of focused attention training (FOCAL) on the duration of novice drivers’ glances inside the vehicle. Ergonomics (54), 917-931. https://pmc.ncbi.nlm.nih.gov/articles/PMC3437545/
11 Arnett, J. (n.d.) Arnett Inventory Sensation Seeking (AISS). https://sjdm.org/dmidi/Arnett_Inventory_of_Sensation_Seeking.html
12 Buss, A. & Perry, M. (n.d.) Buss Perry Aggression Questionnaire (BPAQ). https://psychology-tools.com/test/buss-perry-aggression-questionnaire
13 Zhang, T., Hajiseyedjavadi, F., Wang, Y., Samuel, S., Qu, X., & Fisher, D. (2018). Training interventions are only effective on careful drivers, not careless drivers. Transportation Research Part F (58), 693-707. https://doi.org/10.1016/j.trf.2018.07.004
14 Heinrich Heine Universitat Dusseldorf. (2023). G*Power 3.1 Manual. https://www.psychologie.hhu.de/fileadmin/redaktion/Fakultaeten/Mathematisch-Naturwissenschaftliche_Fakultaet/Psychologie/AAP/gpower/GPowerManual.pdf
15 Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, pages 285-287. https://doi.org/10.4324/9780203771587
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Part B (11-19) |
| Author | Fisher, Donald (Volpe) |
| File Modified | 0000-00-00 |
| File Created | 2025-12-11 |