Testing Plan

Attachment C

CDC Usability and Digital Content Testing: Testing Plan

Goals

The primary goal of this user experience (UX) digital content usability testing package is to assess the usability and customer satisfaction of content on CDC digital media platforms including CDC.gov website, @CDCgov social media platforms and CDC YouTube videos on both desktop and mobile platforms. The purpose is to ensure CDC’s digital communication of text and visual content is clear, understandable and actionable the first time CDC customers see or hear the information.

This package will allow for rapid (within 2-3 weeks) usability and digital content collections to meet CDC's communication needs, including during outbreaks and emergency responses. During emergency responses such as the annual respiratory disease response (e.g., COVID, flu and RSV), rapid results are integral to informing necessary changes to CDC’s web and digital content.

In addition, the ability to rapidly conduct short, 20-minute tests of recently updated content and design elements, helps identify problems quickly and allows for immediate corrections before issues become major problems. Working iteratively allows for continuous and on-going data-driven improvements, leading to a better user experience and higher satisfaction with CDC’s website and digital content.

Conducting these short, 20-minute tests of content and design elements will enable CDC to improve its website and digital content in smaller, faster increments.

When the need arises, such as during an emergency response, to test an element (e.g., new design, improved content), CDC will use one of the following tests at a time to gather reactions from participants:

• Content Feedback usability test, to assess readability of content and ways to improve value

• Design Feedback usability test, to gauge reactions to design elements including colors and layout

• A/B Comparison usability test, presenting different design options to see which option users prefer

• Data visualization usability test, verifying that the visual representation of data accurately conveys the intended information and insights

• As an example, CDC would use the A/B Comparison usability test if two potential designs were being considered for social media messages, to see which option users preferred. On a separate occasion, a Content Feedback usability test would be applied to check that the rewritten content on a CDC website is clear and actionable the first time CDC customers see the information.

Digital Media to be tested

• https://www.cdc.gov/

• @CDCgov on social media platforms such as Facebook, Instagram, X, and YouTube

Participants

Participation in the customer experience digital content assessment is voluntary. Participants will be individuals interested in CDC health topics found on CDC.gov, @CDCgov social media platforms, or CDC YouTube videos selected from an existing panel of participants. A participant can only participate once.

CDC has a subscription to a usability testing tool through our vendor, UserTesting. CDC will recruit participants by requesting them from participant panels via our UserTesting vendor’s usability test tool subscription or website links. UserTesting has created participant panels of people who opt in to participate in future tests. Using their self-reported interests and traits (e.g., age, parental status, web expertise, social media use, etc.), UserTesting assigns the users to panels.

With this subscription, CDC sets up the test in the UserTesting (vendor) site:

1. We request participants from the panel of users across using audience filters and screener questions (e.g., age, United States, web expertise, profession, etc.)

2. UserTesting (vendor) randomly selects participants from their broad panel of 1.6 million users across 30 countries that meets the requested audience and displays a Dashboard message to those users

3. These potential participants see dashboard of all potential tests they can take that includes the amount the UserTesting (vendor) will compensate them (determined by length of a test) and first question. Users then select the test if they wish to participate (see Figure 1).

4. Once the desired number of participants complete the test, the test stops, and our CDC team gets results.

Remote Participation

Participants will remotely access a web-based testing tool while answering questions and completing activities that will be placed on www.cdc.gov or @CDCgov social media platforms with no facilitator present.

Devices

During the assessment, the participants will use a desktop computer or mobile device as well as an internet connection to access the assessment tool.

Consent Form

Participants will digitally sign the consent form (see Attachment D: Consent Form) before the session begins.

Instructions

Instructions will be included in the assessment testing tool (see Attachment E: Instruction and Activities).

Sessions

During each session, the participant will:

• Be asked to sign the consent form.

• Be provided instructions.

• Be asked to provide responses to assessment questions or activities.

Unmoderated assessments are expected to last about 20 minutes.

Activities/Follow up Questions

Participants will be asked a set of questions or activities to complete. (See Attachment E: Instruction and Activities for a list of questions and activities the participant may be asked.) All questions and activities will be provided to the participants in the web-based assessment tool.

2