The Centers for Medicare & Medicaid Services (CMS) is requesting clearance from the Office of Management and Budget (OMB) to implement a national field test of the draft Outpatient and Ambulatory Surgery Experience of Care Survey to assist CMS in finalizing the survey instrument and implementation procedures and in making a decision about possible implementation of the survey on a national scale. This work will be implemented under Contract Number HHSM-500-2012-00087G.
The Outpatient and Ambulatory Surgery Experience of Care Survey field test will be a one-time (cross-sectional) survey of a sample of patients 18 years old and older who received outpatient surgery or a procedure from a HOSD or ASC facility. The field test sample will consist of 4,190 outpatient surgery patients. Eligible patients will include those who received outpatient surgery from hospital outpatient surgery department (HOSDs) or ambulatory surgical centers (ASCs) within the prior month. The sample will be designed to cluster respondents within HOSDs and ASCs.
We will use a two-stage sample design in which the outpatient surgery facilities are selected first, and patients are then selected within the selected outpatient surgery facilities. Our target is to recruit 36 outpatient surgery facilities, comprising 18 HOSDs and 18 ASCs, to obtain 2,304 patient interviews (64 patient interviews per HOSD or ASC).
The facility sampling frame will be constructed from all eligible outpatient surgery facilities recruited through the POS file or through facilities that have volunteered to participate. We will recruit 36 outpatient surgery facilities for the field test, among them 18 HOSD facilities and 18 ASC facilities from multiple states within the U.S. We will carefully balance the outpatient surgery facility sample to represent key facility characteristics and ensure that the outpatient surgery facility sample is also distributed over different regions. The three key facility characteristics are:
Facility Size1: Large, Medium, Small
Facility Location: Urban, Rural
Facility Ownership: Hospital, Management Company, Health System/Managed Care, Physician, Government.
To be eligible to be selected for the Outpatient and Ambulatory Surgery Experience of Care Survey field test sample, patients must meet the following eligibility requirements:
have had an outpatient surgery or procedure from the facility in the prior calendar month;
were at least 18 years old when they received their outpatient surgery or procedure; and
were not discharged to hospice or to a hospital for an in-patient stay.
We will construct a patient list comprised of eligible patients from each selected outpatient surgery facility. To ensure that the patients selected into the sample are representative of the broad range of outpatient surgery procedures, we will stratify the patient sampling frame into four surgery groups (strata), representing the largest categories of surgery types performed based on CMS data from 2011. The four strata include: Ophthalmology, Orthopedics, Gastrointestinal, and all others.
The patient sample will be allocated to the four patient surgery strata proportionally to the number of patients in each stratum. To balance the patient sample by age, gender, and race/ethnicity, the patient sampling frame will be sorted by age group, gender, and race/ethnicity with each patient stratum before selecting a systematic random sample using PROC SURVEYSELECT in SAS.
Our goal is to complete 64 patient interviews from each HOSD and ASC, which is sufficient to have at least 80% power to detect a medium Cohen effect size (D=0.5) difference between two outpatient surgery facilities using a two-tailed test at alpha=0.05. Assuming a 55% response rate, we will select 116 patients from each HOSD and ASC.
The proposed sample size, expected number of completed interviews, and expected response rates for each phase of the mail with telephone follow-up of nonrespondents data collection strategy are presented in Exhibit 4.
Exhibit 3. Proposed Sample Size, Expected Number of Completed Interviews, and Expected Response Rate: Field Test
Field Test |
Sample Size |
Number of Respondents |
Response Rate (%) |
Initial Mail |
4,190 |
1,341 |
32.0 |
Phone Follow-up |
2,849 |
963 |
33.8 |
Overall |
4,190 |
2,304 |
55.0 |
We will use a quasi mixed-mode survey administration approach to implement the field test. All sampled patients will receive an initial mailing of a questionnaire, cover letter, and postage-paid return envelope. Sample patients who do not respond to the mail survey within three weeks after the questionnaire is mailed will be assigned to the telephone follow-up. Telephone interviewers will make up to 5 attempts to complete the interview by phone with all mail survey non-respondents. Data collection will end six weeks after the first questionnaire package is mailed.
To reduce nonresponse bias, every effort will be made to maximize the patient response rate while retaining the voluntary nature of the survey. We expect to achieve a response rate of approximately 55%. Conducting the field test as a mail with telephone follow-up of nonrespondents will improve the response rate by offering sampled patients two different ways to respond to the survey.
The questionnaire mailing will include a cover letter containing information about the survey, including sponsorship and objectives, a description of how survey results will be used, and the name and toll-free telephone number of a survey staff member that sampled patients can contact if they have questions or need additional information about the survey. We will also include a statement that assures patients that their survey responses will not be linked to their names or any other information that can identify them.
Following best mail survey practices, we will use an easy-to-read font no smaller than 12 point size in the survey cover letter and questionnaire, allowing ample white space between questions in the questionnaire, avoiding a format that displays the questions as a matrix, using a unique subject identification number on the questionnaire rather than printing the sample member’s name, and displaying the OMB number and expiration date on the questionnaire (Dillman, 1999).
For the telephone follow-up of mail survey nonrespondents, we will make up to 5 attempts to reach each sample patient, with those attempts varying by day of the week and time of day. Telephone interviewers will be trained on how to answer questions that are most frequently asked by sample patients and to address any concerns that they may have about participating in the survey.
The field test will test survey administration procedures that have been used on other CAHPS patient experience surveys, including a mailout of a questionnaire followed by telephone follow-up contacts to mail survey nonrespondents.
This sampling and statistical plan was prepared by RTI International. The primary statistical design was provided by Patrick Chen of RTI International. Mr. Chen can be reached by telephone at (919) 541-6309 or by e-mail at pchen@rti.org.
Dillman, Don A. (1999). Mail and Other Self-Administered Surveys in the 21st Century: The Beginning of a New Era. The Gallup Research Journal, 2(1), 121–140.
U.S. Bureau of Labor Statistics (2010). National Compensation Survey: Occupational wages in the United States, July 2010. U.S. Department of Labor, Bureau of Labor Statistics. Available at http://www.bls.gov/ncs/ncswage2010.pdf. Based on average wages.
1 Ideally facility size can be defined according to the number of patients a facility served from the POS file. Facilities will be sorted by patient volume; the top 1/3 of facilities will be classified as ‘Large’, the middle 1/3 of facilities as ‘Medium’, and the bottom 1/3 of facilities as ‘Small’. However, if the information of the number of patients a facility served is unavailable, we could use other measures as a surrogate, such as the number of Medicare claims, number of operating rooms, etc.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | cannada |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |