FY 2010 Field Office Caller Survey Description

DOCUMENTATION FOR THE GENERIC CLEARANCE

OF CUSTOMER SATISFACTION SURVEYS

TITLE OF INFORMATION COLLECTION: Fiscal Year (FY) 2010 Field Office (FO) Caller Survey

SSA SUB-NUMBER: F-03

DESCRIPTION OF ACTIVITY (give purpose of activity, provide specific information; i.e., date(s) of survey, number of focus groups, locations, etc.):

BACKGROUND

The Social Security Administration (SSA) Service Delivery Feedback Program mandates that we annually survey individuals who do business with SSA to assess their satisfaction with the specific mode of contact they used. As part of this mandate, we are conducting the FY 2010 FO Caller Survey, in which we will interview members of the public who contacted an SSA FO by telephone to transact their business. We designed the survey, which takes place shortly after the participant’s FO call, to 1) measure public perception of SSA’s FO telephone service and 2) collect information that will assist us in understanding the customer experience behind satisfaction ratings. SSA has been conducting a satisfaction survey with this population since 1997.

SURVEY

Description of Survey

SSA will conduct the FO Caller Survey over a 4-week period beginning in April 2010. We conduct the survey during the same general timeframe each year to ensure comparability of results. An SSA contractor will conduct the survey interviews by phone. The survey covers a variety of topics relating to SSA’s FO caller service, including the following:

• Satisfaction with how quickly the caller got through to the FO;

• Whether the person had to make more than one call that day to reach the office;

• Ease of understanding of the FO’s automated phone message;

• What action the person took after hearing the FO’s automated phone message;

• Whether the person left a voice mail message and if they received a return call;

• Whether the person was connected immediately or had to wait on hold for an agent;

• Satisfaction with the time spent on hold waiting to speak to an agent;

• Satisfaction with the courtesy, job knowledge level, and helpfulness of the agent;

• What happened to callers who were unable to get served that day;

• If the person was unable to complete their business on the phone during their call, which contact method they would use to do so;

• The person’s overall rating of the FO telephone service;

• The person’s service preference for future business;

• The type of business the person was calling about; and

• The kind of benefits the person was calling about (if applicable).

During FY 2009 SSA, began a phased-in installation of a centralized Voice-over-Internet Protocol (VOIP) system in 1,300 local FOs. That effort will continue during FY 2010. To better capture caller perceptions of the new service using the VOIP equipment, we made revisions to the traditional FO Caller Survey questionnaire. Although we changed the order in which some questions are presented to participants, 19 questions in the FY 2010 survey are identical to those used in prior surveys. We made minor wording changes to four questions to improve clarity. And finally, we added one new question (#15) to facilitate the way we identify how callers were served and directed to the proper follow-up question.

We successfully conducted a pilot survey in the first quarter of FY 2010 (the FO Telephone System Replacement Project Survey) to test the new version of the questionnaire. We were able to monitor live survey interviews to evaluate participants’ responses to the survey questions. We made some minor wording changes to several survey questions for the annual survey as a result of the live monitoring.

Statistical Information

Sample Selection

From the group of all FOs in which the new VOIP phone system has been installed, we will select a random sample of 50 FOs to participate in the survey. The mix of sampled offices will represent all 10 SSA regions and all types of FOs. During the 4‑week sample selection period, we will process the recorded call data to identify completed calls representing individuals who “got through” to these 50 FOs.

The 50 FOs selected for the survey receive approximately 3 million calls each year. Based on these estimates, we anticipate that over the 4-week period of the survey they will receive a total of 250,000 calls. From this universe of calls received, we will randomly select 4,000 unique telephone numbers. We will use these originating numbers to attempt to contact and interview the individuals who placed the sample calls. During the interview process, we will screen out calls that originate from a public or business telephone, since we cannot usually identify individuals who call FOs from such locations.

Methodology

As we mentioned above, SSA-approved contractors will conduct this survey by telephone and will translate the survey questionnaire into computer-assisted interviewing (CATI) software so that interviewing and data input of the individual’s responses occur simultaneously. The CATI software also ensures that only questions pertinent to the individual’s circumstances are asked, allowing for complicated question paths while still minimizing response burden.

Response Rate

To maximize the response rate for this survey, SSA will:

• Select samples twice a week (to minimize the time between the customers’ service experience and the survey)

• Transmit sample listings to the contractor within a week of the FO call;

• Have the contractor make at least 15 attempts to reach the caller over a 3‑week period in an effort to obtain the highest possible response rate;

• Have the contractor arrange call attempts for daytime, evening, and weekend hours;

• Have the contractor arrange call backs and leave messages for the sampled individual as necessary, and provide a toll-free number for return calls to encourage participation;

• Proved Spanish-speaking interviewers as necessary.

In the FY 2008 FO Caller Survey we achieved a response rate of 41 percent, somewhat lower than the 50 percent response rate we anticipated. We discussed the issue with our contractor who improved survey oversight and conducted more thorough interviewer training prior to the start of the FY 2009 survey. As a result of these efforts, our contractor reported a 6 percentage point increase in the response rate (47 percent) for the FY 2009 survey. We anticipate that we will achieve a similarly good response rate in the FY 2010 FO Caller Survey. Fifty percent is the highest response rate to be expected given that the only contact information we have is the incoming telephone number.

Sampling Variability

The key variable in the FO Caller Survey is overall satisfaction with telephone service. In the FY 2009 FO Caller Survey, overall satisfaction with FO telephone service was rated 79 percent excellent, very good or good. Our proposed sample size of 4,000 is large enough to allow for the screening of business calls and, assuming a 50 percent response rate from the remaining sample, will provide a sampling variability at the 95-percent confidence level of +/‑2.1 percent for this key indicator of satisfaction.

SSA’s Office of Quality Performance (OQP) will perform all sampling and data analysis. Dan Zabronsky, Director of Modeling, Office of Quality Data Management, SSA, will provide statistical support. OMB can reach him at (410) 965-5953.

IF FOCUS GROUP MEMBERS WILL RECEIVE A PAYMENT, INDICATE AMOUNT (No more than $25 can be authorized under OMB rules):

N/A

USE OF SURVEY RESULTS:

SSA will use the results of this study to evaluate and improve various aspects of its FO telephone service. We will also combine the results of the Field Office Caller survey with the findings from the 800 number and Office Visitor surveys to report a combined performance measure for SSA each year: “percent of the people doing business with SSA who rate overall service excellent, very good or good.”

BURDEN HOUR COMPUTATION (Number of responses (X) estimated response time

(/60) = annual burden hours):

Number of Responses: 4,000

Estimated Response Time: 10 minutes

Annual Burden Hours: 667 hours

NAME OF CONTACT PERSON: Debbie Larwood

TELEPHONE NUMBER: 410-966-6135

Generic Clearance FO Caller Survey

February 2010

4