BACKGROUND
The Centers for Medicare & Medicaid Services (CMS) is requesting clearance from the Office of Management and Budget (OMB) to implement nationally the Consumer Assessment of Healthcare Providers and Systems Outpatient and Ambulatory Surgery Survey (OAS CAHPS) to measure patients’ experience of care with outpatient and ambulatory surgery centers under Contract Number HHSM-500-2014-00426G. Under this information collection request, CMS is also seeking review and approval for a mode experiment that will be conducted to determine the impact of using three different data collection modes in the national implementation of OAS CAHPS.
Since 1995, the Agency for Healthcare Research and Quality (AHRQ) and its Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Consortium, in conjunction with CMS, have developed standardized CAHPS Surveys and tools for a variety of patient populations, including commercially insured ambulatory patients, patients whose care is covered by Medicare and Medicaid, dialysis patients, home health patients, hospital inpatients, dental patients, and patients who receive behavioral health care and services. The purpose of the CAHPS family of surveys is to collect data about patients’ assessment and rating of the care they receive from their health care provider or health care system.
CMS began looking at Medicare beneficiaries’ experience of care with outpatient surgery via the Patient Satisfaction with Outpatient Survey: A National Survey of Medicare Beneficiaries conducted by the U.S. Office of the Inspector General in 1989. However, until recent years, the focus has been on patients’ experience with other types of health care. In 2006, CMS began implementing the Hospital CAHPS (HCAHPS) Survey, which collects data about hospital inpatients’ rating of and experience with hospital inpatient care. CMS began publicly reporting HCAHPS Survey results on the Hospital Compare link on the Medicare.gov website in 2008. The HCAHPS Survey, however, includes data from samples of patients who receive inpatient hospital care. It does not include patients who received outpatient surgical care from hospital-based outpatient surgical departments (HOPDs), nor does it include patients who receive outpatient surgery from independently owned, freestanding ambulatory surgical centers (ASCs). In 2009, AHRQ reviewed and approved a Surgical Care CAHPS (S-CAHPS) Survey, the development of which was sponsored by the American College of Surgeons and the Surgical Quality Alliance. However, that survey, which is applicable to patients who receive surgery in inpatient and outpatient settings, focuses on care provided by a specific surgeon, not on the care received at the facility level. An instrument measuring patient experience at the HOPD/ASC facility level would include the care, communication, and consideration provided by all physicians, nurses, and other facility staff that a patient might encounter before, during, and after an outpatient surgery visit. Additional topics include facility cleanliness, postsurgical care coordination and follow-up, and patient-reported outcomes.
Although some HOPDs and ASCs are conducting their own patient experience of care surveys with patients who receive outpatient surgery, to date there does not exist a single standardized survey instrument in use to assess patient experiences with outpatient surgical care in HOPDs or ASCs. CMS believes that it is necessary to have a single standardized survey instrument whose results can be compared across facilities, rather than having separate individual facility-sponsored surveys. While there is no requirement at the present to implement and report experience of care data, CMS is establishing a voluntary reporting program in which ASCs and HOPDs can choose to report their survey data to CMS and also choose whether or not to publically report data.
In recent years, the number of ASCs has increased considerably. According to the Medicare Payment Advisory Commission’s 2014 Report to Congress on Medicare Payment Policy (MedPac, 2014), the number of Medicare-certified ASCs has grown by 12% from 2007 through 2012, increasing from 4,798 in 2007 to 5,357 by the end of 2012. That same report also shows that Medicare payments to ASCs have increased by 24% over that same time period, from $2.9 billion in 2007 to $3.6 billion in 2012. On the other hand, on average, Medicare spending on outpatient services grew from $29 billion in 2006 to $46 billion in 2011, which equates to 7.8% (MedPac, 2014). The Ambulatory Surgery Center Association (ASCA) also notes an increased volume of ASCs and HOPDs from approximately 5 million and 9.5 million (respectively) in 2005 to 6 million and 11 million (respectively) in 2010 (ASCA, 2011). Considering the growing number of ASCs and the increasing Medicare expenditures for outpatient surgical services from ASCs and HOPDs, the implementation of OAS CAHPS will provide CMS with much-needed statistically valid data from the patient perspective to inform quality improvement and comparative consumer information about specific facilities.
A rigorous survey development process was undertaken to create the finalized instrument. In 2012, CMS partnered with a contractor (RTI International) to develop and test a standardized survey to measure the experiences of patients who receive outpatient surgical care from hospital-based outpatient surgical departments and independently owned ambulatory surgery centers. Prior to the pilot testing phase, significant background research was conducted, including the following steps:
Literature Review. An exhaustive literature review was performed, gathering information on outpatient surgery surveys and issues to inform the development of the survey or protocols for implementation. The purpose of the literature review was to obtain information about the dimensions of domains of care that may be of interest to consumers when choosing an outpatient surgery center and issues that would affect the development of the survey. In addition, the literature review was used to identify other issues that would affect the development of the survey, including sampling approaches, data collection, reporting, and quality improvement issues.
TEP Input. RTI convened a meeting with a Technical Expert Panel (TEP) comprising individuals from industry, professional associations affiliated with the outpatient surgery industry, and academia in February 2013. The purpose of the meeting was to discuss the goal of the survey and to understand how to encourage facilities to participate in the field test. The TEP was also given the opportunity to provide feedback on the focus and content of the survey. The team also met and communicated with the CAHPS Consortium multiple times during in the various stages of the survey development process to solicit feedback.
Federal Register Notice. On January 25, 2013, CMS published a Federal Register Notice soliciting the submission of survey domains and topic areas in the public domain measuring outpatient surgery patients’ experience of care. The notice of request for measures closed on March 26, 2013.
Review of Submitted and Existing Instruments. RTI reviewed all of the responses for their relevance for inclusion in this survey. Submitted items were entered into a comprehensive database, allowing comparisons across domains and topic areas. Other existing CAHPS Survey instruments that are publicly available were also reviewed for relevancy. Individual items from both the submitted and other existing CAHPS Surveys were examined by the team’s methodologists for possible inclusion in the OAS CAHPS draft questionnaire.
Public Comment. In early October 2013, another Federal Register notice was published seeking public comment on the draft instrument and protocol. CMS received two comments by the end of the 60-day window (in early December 2013). The 30-day notice was published in late December 2013 and did not produce any additional feedback. The additional feedback received through this process was also used to refine the survey instrument.
Field Test. RTI conducted a 6-week field test of OAS CAHPS in July and August 2014 to test the reliability and validity of the survey items and implementation procedures. Survey participants included patients who had a recent outpatient surgery (in May 2014) at one of the participating facilities. Patients who had a recent diagnostic procedure, such as a colonoscopy were also eligible. The survey questionnaire that was tested contained questions about the check-in process, facility environment, patient’s experience communicating with administrative staff (receptionists) and clinical providers (doctors and nurses), attention to comfort, pain control, provision of pre- and postsurgery care information, overall experience, and patient characteristics. The field test included 4,179 sampled patients from a total of 36 facilities (18 HOPDs and 18 ASCs) located across the United States.
The field test was implemented as a mixed-mode design (i.e., an initial mailed questionnaire followed by a telephone follow-up of nonrespondents to the mail survey) allowing us to test procedures for both mail and telephone survey administration (via computer-assisted telephone interview [CATI]) in English and Spanish. The data collection period spanned 6 weeks. The first phase (mail) lasted three weeks and the second phase (telephone follow-up to nonrespondents) lasted 3 weeks.
Of the 4,179 sample patients, 1,863 responded to the survey resulting in an overall (adjusted) response rate of 45.61%. Of those respondents who completed the survey, 30.4% responded by mail and 13.8% responded by telephone.
Field Test Analysis. The core of our field test analysis was a psychometric analysis (including tests of reliability and validity) of the survey items and proposed reporting composites. The goal of such an analysis was to assess the measurement properties of the proposed instrument and sub-domain composites created from item subsets, to ensure that the information reported from any future administrations of the survey was well-defined. Such careful definition will prevent data distortion or misinformation if they are publicly reported.
Final Survey. Based on the field test findings, the survey instrument was revised. Twelve questions were removed from the 49-item instrument. The final survey has 37 items.
This OMB submission is in support of the national implementation of OAS CAHPS on a voluntary basis for HOPDs and ASCs. Starting in 2015, HOPDs and ASCs will be invited to participate in a mode experiment conducted by CMS’s contractor. The outpatient surgery centers will be asked to provide data to the contractor that will, in turn, collect and submit to CMS the OAS CAHPS data on their behalf. Starting in 2016, HOPD and ASC facilities will be invited to participate in the national implementation of OAS CAHPS by contracting with a CMS-approved, independent third-party survey vendor to implement the survey on their behalf and to submit the OAS CAHPS data to CMS.
As discussed above, CMS’s contractor conducted a field test of OAS CAHPS in 2014 to test the reliability and validity of the survey items and implementation procedures. The field test was conducted in both English and Spanish. After reviewing field test results with the TEP, OAS CAHPS was finalized as a 37-item survey instrument. The final version of OAS CAHPS can be found in Attachment A.
The OAS CAHPS instrument contains questions about the patient’s experience with the check-in process, facility environment, communication with administrative staff (receptionists) and clinical providers (doctors and nurses), attention to comfort, pain control, provision of pre-and postsurgery care information, overall experience, and patient characteristics. The patient characteristic questions (within the “About You” section) were designed to comply with the U.S. Office of Minority Health’s requirements on data collection standards for race, sex, ethnicity, primary language, and disability status. Patients will also be asked to provide overall ratings of the outpatient surgery facility.
For national implementation, we will follow a similar model to other existing CAHPS Surveys in that we will require that facilities contract with a third-party, CMS-approved survey vendor to administer OAS CAHPS. Because data from the national implementation of OAS CAHPS will be used to produce comparative results, and because the national implementation of the survey will be conducted by multiple independent CMS- approved survey vendors, it is important that all vendors administer the survey using the same survey administration protocols and specifications. Therefore, CMS-approved survey vendors conducting OAS CAHPS on behalf of the HOPDs and ASCs will be required to use survey administration specifications developed by CMS. The HOPDs and ASCs will be able to choose a vendor to administer the survey using one of three data collection modes: mail-only, telephone-only, and mixed mode (mail survey with telephone follow-up of nonrespondents).
CMS will begin publicly reporting comparative results from OAS CAHPS after each facility has conducted data collection for 12 months. OAS CAHPS measures will enable consumers to make more informed decisions when choosing an outpatient surgery facility, aid facilities in their quality improvement efforts, and will help CMS monitor the performance of outpatient surgery facilities.
In order for patients to make objective and meaningful comparisons between outpatient surgery facilities, methods and adjustments must be put into place to account for significant sources of bias outside the control of the facilities. Known sources of bias include data collection mode and variability in patient-mix and response propensity across patients within outpatient surgery facilities. As part of this information collection request, CMS is requesting approval to conduct a randomized mode experiment with a sample of patients receiving outpatient surgeries or procedures to determine whether they respond differently to the survey based on data collection mode. In addition, data from the mode experiment will be used to determine which, if any, patient characteristics affect the ratings and reported experience of the healthcare encounter. If needed, during the national implementation of OAS CAHPS, CMS will develop models to statistically adjust survey results before comparative results are publicly reported. Comparative results from the OAS CAHPS mode experiment will not be publicly reported.
CMS is working with RTI to recruit approximately 100 outpatient surgery centers (50 HOPDs and 50 ASCs) to participate in the mode experiment. The mode experiment will involve up to 13,737 patients across the three survey modes (mail-only, telephone-only, and mixed-mode). To ensure that the patients selected into the sample are representative of the broad range of surgery procedures, we will sort the patient sampling frame by gender, age group, and procedure type, prior to selecting the patient sample.
The information collected in the national implementation of OAS CAHPS will be used for the following purposes:
To provide a source of information from which selected measures can be publicly reported to beneficiaries to help them make informed decisions for outpatient surgery facility selection;
To aid facilities with their internal quality improvement efforts and external benchmarking with other facilities; and
To provide CMS with information for monitoring and public reporting purposes.
The national implementation of OAS CAHPS is designed to allow third-party, CMS-approved survey vendors to administer OAS CAHPS using mail-only, telephone-only, or mixed-mode (mail with telephone follow-up). Experience from previous CAHPS surveys shows that mail, telephone, and mail with telephone follow-up data collection modes work well for respondents, vendors, and health care providers. Any additional forms of information technology, such as web surveys, would be less feasible with OAS CAHPS patients, as patient e‑mail address information is not readily available through HOPDs and ASCs.
The CMS-approved survey vendors who administer the survey during the national implementation will use an electronic data collection or CATI system if they administer a telephone-only or mixed-mode survey. CATI will also be used for telephone follow-up with mail survey nonrespondents during the mode experiment. There are numerous advantages to administering a telephone interview using a CATI system, including the following:
costs less than in-person data collection;
allows for a shorter data collection period;
allows for less item nonresponse because the system controls the flow of the interview and complex routing;
increases data quality by allowing consistency and data range checks on respondent answers;
creates a centralization of process/quality control; and
reduces post-interview processing time and costs.
OAS CAHPS was designed to collect information that is fundamentally different from other CAHPS or patient experience of care surveys. CMS is not aware of any existing validated survey instrument where the unit of analysis is the hospital outpatient surgery department or ambulatory surgery facility, and the focus of the survey is on patient-reported experience of care. The information collected through this survey will therefore not duplicate any other effort and is not obtainable from any other source.
Many HOPDs and ASCs are already carrying out their own patient experience of care surveys. These diverse surveys do not allow for comparisons across outpatient surgical facilities. Making comparative performance information available to the public can help consumers make more informed choices when selecting an outpatient surgery facility and can create incentives for facilities to improve care they provide. OAS CAHPS will provide a standardized tool for collecting such information, comparisons across all facilities to enable consumers to compare facilities.
The current version of OAS CAHPS consists of a core set of questions followed by the mandated “About You” questions. In addition, outpatient surgery facilities may include their own facility- specific questions to the existing OAS CAHPS as long as these appear after the core survey questions.
Survey respondents are patients who have received care from a hospital-based outpatient surgery center or independently owned ASC. The survey should not impact small businesses or other small entities.
CMS will conduct the OAS CAHPS mode experiment as a one-time (cross-sectional) survey of patients currently receiving care from HOPDs and ASCs. A random sample of eligible patients who received outpatient surgical care during the preceding month will be selected and contacted to gather data about their experiences with outpatient care. CMS will construct the sample frame for the mode experiment using patient information provided by the participating HOPDs/ASCs. Data collection for the mode experiment will take place in the fall of 2015 or immediately after OMB approval for this information collection request is received. Conducting this one-time survey prior to the national implementation of OAS CAHPS is crucial for determining differences in survey responses based on the three modes of data collection and to determine patient characteristics that might affect experiences and the ratings of the care they receive. The mode experiment findings will produce estimates for any patient mix and mode adjustments that might be necessary to publically report comparative information.
The national implementation of OAS CAHPS on a regular basis will allow for the collection of data about patients’ experience with outpatient surgical care at different points in time during a calendar year. Regular implementation will also allow sampled patients to assess their experience at the facility soon after their surgery or procedure is performed. Participating facilities will be asked to provide a sample frame consisting of patients who received at least one surgery or procedure during the sample month to their survey vendor on a monthly basis. Vendors will, in turn, be required to initiate the data collection from patients within 3 weeks after the sample month closes. Respondent burden is increased and the recall factor becomes a problem if patients are asked to recall their care experiences after longer lapses of time. Monthly sampling and administering the survey within 3 weeks after the close of the sample month will reduce the amount of time between outpatient care event and survey. Respondent recall will be enhanced, thus improving the quality of survey data and results. For this reason, CMS does not believe that a less frequent data collection period will result in the most accurate and complete data for public reporting and quality monitoring purposes. Although data collection will be completed by vendors on a monthly basis, data will be submitted on a quarterly basis.
There are no special circumstances with this information collection request.
The 60-day Federal Register notice published on January 16, 2015 (80 FR 2430). A total of 10 comments were received on or before March 17, 2015. As a result of one of the questions about the cost to facilities, Section A.13 has been updated to reflect a revised assumption about the cost for securing a vendor for the National Implementation. No other comments had any impact on the implementation plans or the survey content.
Information about OAS CAHPS was initially posted in the October 4, 2013 (78 FR 61848), Federal Register.
For the survey development, CMS’s contractor convened a 10-member TEP and obtained guidance and input from the TEP on the sample design and survey administration specifications for both the national implementation and mode experiment of OAS CAHPS. The TEP members consulted represented the following organizations:
Anesthesia Quality Institute;
ASC Quality Collaboration;
University of North Carolina at Chapel Hill;
Cleveland Clinic Health System;
Carilion Clinic Orthopedics;
National Center for Health Statistics;
HONOReform;
Ohio State Government;
The Joint Commission; and
Trinity Surgery Center.
No payments or gifts will be provided to respondents.
Individuals contacted as part of this data collection will be assured of the confidentiality of their replies under 42 U.S.C. 1306, 20 CFR 401 and 422, 5 U.S.C. 552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974), and OMB Circular A-130. The participant will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose. However, in instances where respondent identity is needed, the information collection will fully comply with all aspects of the Privacy Act.
Concern for the confidentiality and protection of respondents’ rights is critically important on any patient experience of care survey. Some patients might not be willing to participate in the survey for fear of retribution from the facility, Medicare, Medicaid, or other payer. There is also a concern that some patients might respond to the survey but might respond in a way that does not reflect their actual experiences with outpatient surgical care. Therefore, assurances of confidentiality are critically important with this patient population.
OAS CAHPS patients will be more willing to participate if an outside organization administers the survey. In addition, the HOPDs and ASCs will be asked not to discuss OAS CAHPS with their patients, and especially in any way that might influence the patients’ decision to participate in the survey or their responses to the survey. The cover letter included with the mail survey questionnaire sent to sample patients will encourage patients to call the survey vendor’s toll-free telephone number if they have any questions about the survey.
During the mode experiment, HOPDs and ASCs will provide RTI with an electronic file containing information about patients who received an outpatient surgery or procedure in a designated month. The information will include personally identifiable information (name, address, date of birth, and telephone number). The file will also include protected health information used for survey eligibility determination and patient-mix analysis, including CPT-4 codes (indicator of types of surgeries or procedures performed), date of surgery/procedure, and facility name. Social Security numbers will not be provided.
RTI will include an assurance of confidentiality of the data to all participants. For those participating in the mail survey, the cover letter will include an assurance of confidentiality. The introductory script for those interviewed by telephone will contain an assurance of confidentiality. Those patients included in the mixed-mode data collection could receive the assurance of confidentiality twice—in the mail cover letter, and in the telephone introductory script should the phone interview be necessary (the mail survey cover letters and the telephone interview introductory script are included in Attachments B and C, respectively).
RTI understands the privacy and confidentiality concerns regarding access to OAS CAHPS data. All RTI staff members who will have access to patient information will be required to sign and abide by the terms of a nondisclosure agreement, where they agree to protect the identity of patients included in the mode experiment and the data they provide. RTI has redundant security protocols to protect data and computer systems. Servers are maintained in climate-controlled environments, with restricted access. A firewall stands between the internal systems and the Internet, requiring authentication of all users requesting access. User identification and passwords are unique and changed on a regular basis. Full backups are conducted on a weekly basis, with incremental backups performed nightly. Copies of backup materials are stored offsite in a secure location in case of system failure.
As data are collected and assembled into databases for analysis and interpretation, RTI incorporates a number of database security safeguards to protect data from accidental or intentional access and disclosure threats. RTI’s data collection and storage security measures include the following:
Maintenance of all servers in RTI’s environmentally controlled Computer Center, where computers are located in a center constructed of masonry with an automatically locking steel door that is locked at all times; fire protection is provided by a halon system with all servers having an Uninterruptible Power Supply.
User ID and password authentication to access all systems. Where appropriate, systems are configured to support the use of Digital Security Certificates for additional user authentication.
Encrypted transmission of data.
Use of Transport Layer Security, the successor technology to Secure Socket Layer for encryption of data across the Internet.
Connection to the Internet by an Internet firewall via a high-speed T2 (6.2 MBs) line. In the event of a failure, a T1 (1.544 MBs) backup will automatically provide uninterrupted Internet connectivity. Subscription to virus-protection services from McAfee VirusScan with automated update of virus signature files on all computers.
Redundant servers with automatic switchover to ensure 24/7 availability.
Daily incremental backups of all data files, with full backups created weekly.
Offsite storage of data backups.
For the national implementation, survey vendors approved to conduct OAS CAHPS for participating facilities will be required to have systems and methods in place to protect the identity of sampled patients and the confidential nature of the data that they provide. The survey vendors will submit only de-identified survey data to RTI for analysis.
OAS CAHPS vendors will be required to include the following assurances of confidentiality in communications with sample patients:
the purposes of the survey and how survey results will be used;
participation in OAS CAHPS is voluntary;
the information they provide is protected by the Federal Privacy Act of 1974 (and that all project staff have signed affidavits of confidentiality and are prohibited by law from using survey information for anything other than this research study);
their survey responses will never be linked to their name or other identifying information;
all respondents’ survey responses will be reported in aggregate, no facility will see their individual answers;
they can skip or refuse to answer any question they do not feel comfortable with; and
their participation in the study will not affect the outpatient care or Medicare benefits they currently receive or expect to receive in the future.
Information collected in this survey is not considered to be of a sensitive nature. Questions in the survey are confined to respondent interactions and experiences with their outpatient surgery facility.
There is no cost to respondents other than approximately 8 minutes of their time to complete the survey. The estimated 8-minute length of the survey (.13 hours) is based on timing analysis from the field test data collection. The Bureau of Labor Statistics (BLS) reported that the average hourly wage for civilian workers in the United States in 2013 (U.S. Bureau of Labor Statistics, 2013) was $22.33. An estimate of $23 per hour allows for inflation and represents a conservative estimate of the wages of respondents. Estimated annualized burden hours and costs to the respondent for the mode experiment are shown in Exhibits A‑1 and A‑2. The annual total cost burden is estimated to be $14,076.
Exhibit A-1. Estimated Annualized Burden Hours: Mode Experiment
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Hours per Response |
Total Burden Hours |
OAS CAHPS (mail-only, telephone-only, and mixed-mode) |
4,710 |
1 |
.13 |
612 |
Exhibit A-2. Estimated Annualized Cost Burden: Mode Experiment
Form Name |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage Rate* |
Total Cost Burden |
OAS CAHPS (mail-only, telephone-only, and mixed-mode) |
4,710 |
612 |
$23.00 |
$14,076 |
* Based on average wages, “May 2013 National Occupational Employment and Wage Estimates United States” U.S. Department of Labor, Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes_nat.htm#00-0000).
As with the mode experiment, there is no cost to respondents other than spending approximately 8 minutes of their time to complete the survey. Estimated annualized burden hours and costs to the respondent for the national implementation are shown in Exhibits A‑3 and A‑4. These estimates assume that 9,363 HOPDs and ASCs (the universe of Medicare-certified facilities) will sponsor OAS CAHPS and that 300 patients sampled from each agency will complete the survey. The estimated count of Medicare-certified ASCs is assumed to be 5,357.1 The estimated count of HOPDs is assumed to be 4006.2 While not all facilities will participate in the national implementation, we have estimated the maximum burden possible.
Exhibit A-3. Estimated Annualized Burden Hours: National Implementation
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Hours per Response |
Total Burden Hours |
OAS CAHPS (mail-only, telephone-only, and mixed-mode) |
2,808,900 |
1 |
.13 |
365,157 |
Exhibit A-4. Estimated Annualized Cost Burden: National Implementation
Form Name |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage Rate* |
Total Cost Burden |
OAS CAHPS (mail-only, telephone-only, and mixed-mode) |
2,808,900 |
365,157 |
$23.00 |
8,398,611 |
* Based on average wages, “May 2013 National Occupational Employment and Wage Estimates United States” U.S. Department of Labor, Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes_nat.htm#00-0000).
The costs to ASC and HOPD facilities for the national implementation will be determined by the selected data collection mode (mail, telephone, or mixed mode) and by the number of sample patients included in the facility sample. The cost to the government for CMS’s OAS CAHPS contractor to coordinate the national implementation of the OAS CAHPS implementation is $1,339,243.
At this time, there is no mandated HOPD and ASC participation in OAS CAHPS; therefore, this survey will be administered by facilities on a voluntary basis. The burden associated with this is the time and effort put forth by the HOPDs and ASCs to submit the OAS CAHPS patient files to their approved OAS CAHPS survey vendor. Each facility participating in OAS CAHPS must prepare and submit to its survey vendor a file containing patient data on patients served the preceding month that will be used by the survey vendor to select the sample and field the survey. Preparing this file (essentially the sampling frame) for most HOPDs and ASCs can vary by the required level of effort. The data elements needed on the sample frame will be kept at a minimum to reduce the burden on all participating facilities. The burden associated with this survey administration is the time and effort put forth by the facility to prepare and submit the file containing patient data on patients. The survey instrument and procedures for completing the instrument are designed to minimize burden on all respondents. No significant burden is anticipated for small facilities beyond providing their contracted vendor with sample files of patients served. We have determined that the provision of the files will take 34 hours for each HOPD/ASC annually. Additional, one-time start-up activities for facilities include contracting with an approved survey vendor and authorize the vendor on the OAS CAHPS website.
CMS believes that the 34 hours of labor that the HOPD/ASC will need to do annually can be conducted by a Database Administrator. The U.S. Bureau of Labor Statistics has determined that the hourly wage of a Database Administrator is $34.51. Therefore, the annual cost of the wage labor would be 34 hours times $34.51 equals $1,173.34 per facility. The total cost for all facilities participating in the mode experiment would therefore be 100 facilities times $1,173.34 equals $117,334. The total cost for all facilities participating in the national implementation would therefore be 9,363 facilities times $1,173.34 equals $10,985,982. In Exhibit A‑5, we have summarized the estimated cost burden to the facilities. If all 9,363 facilities participate in the OAS CAHPS, at the estimated cost of $1,173.34 for contract costs, then the total cost for the mode experiment and national implementation is estimated to be $10,987,155.
Exhibit A‑5. Estimated Cost Burden to Facilities for Preparing Patient Records
Form Name |
Number of Respondents |
Total Burden Hours |
Average Cost to Contract |
Total Cost Burden |
HOPDs/ASCs Patient Records for Mode Experiment |
100 |
34 |
$34.51 |
$1,173.34 |
HOPDs/ASCs Patient Records for National Implementation |
9363 |
34 |
$34.51 |
$10,985,982 |
Total |
9463 |
321,742 |
$34.51 |
$10,987,155 |
The only cost is that for the time of the respondent. There is no anticipated recordkeeping burden because respondents are not required to keep a copy of the survey.
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. We have determined that there is an annual-time cost to the HOPDs/ASCs to secure the services of approved OAS CAHPS survey vendors to conduct OAS CAHPS on their behalf. In Exhibit A‑6, we have summarized the estimated cost burden to the facilities for securing the services of a survey vendor. If all 9,363 facilities participate in the OAS CAHPS, at the estimated cost of $4,000 for contract costs, then the total cost is estimated to be $37,452,000.
Exhibit A‑6. Estimated Cost Burden to Facilities for Survey Vendors for National Implementation
Form Name |
Number of Respondents |
Total Burden Hours |
Average Cost to Contract |
Total Cost Burden |
HOPDs/ASCs contracting with approved OAS CAHPS Survey Vendors |
9363 |
NA |
$4000 |
$37,452,000 |
Total |
9363 |
NA |
$4000 |
$37,452,000 |
The total cost for the contractor to develop the survey administration procedures is $3,894,081 over a period of 30 months or 1,557,632.4 (annual). These costs include recommending and implementing the survey administration procedures; technical assistance for vendors and vendor oversight; and vendor training program including vendor approval process. Other activities include designing a data submission infrastructure, survey management tool, and project website; preparing for and conducting the mode experiment and its analytical activities; preparing public reporting formats.
The purpose of the 2014 package was to field test the survey’s reliability and validity of the survey items and implementation procedures. The field test has been completed. This 2015 package sets out new burden to conduct a mode experiment, to implement nationally, and for the prep of patient records.
Data collection for the voluntary national implementation of OAS CAHPS is scheduled to begin in 2016 by vendors sponsored by the HOPDs and ASCs that wish to voluntarily participate in the survey. Sampling and data collection will be conducted on a monthly basis by survey vendors working under contract with the participating HOPDs and ASCs. Patients who meet survey eligibility criteria (are 18 years or older, received an outpatient surgery or diagnostic procedure in the designated sampled month at a participating ASC or HOPD, and not discharged to hospice) will be eligible for inclusion in OAS CAHPS.
CMS will begin publishing results from the national implementation of OAS CAHPS after each participating outpatient surgery facility has submitted data for 12 consecutive sample months. Survey vendors will submit data to CMS’s OAS CAHPS Data Center (maintained and operated by CMS’s OAS CAHPS contractor) by an established data submission deadline for each calendar year quarter. The OAS CAHPS results that will be publicly reported will reflect one year’s worth of data. In each quarterly data submission, we will adjust the survey results for mode of survey administration, patient mix, and nonresponse, if necessary. The results posted on http://www.Medicare.gov/.
This will be developed as part of this contract, following the mode experiment effort.
As previously mentioned, the OAS CAHPS mode experiment will be a one-time (cross-sectional) survey of a sample of patients 18 years old and older who received outpatient surgery or a procedure from a HOPD or ASC facility during the sample month. The mode experiment sample will consist of 13,737 patients. Data collection for the mode experiment will take place in the fall of 2015 or as soon as possible after OMB clearance is received. Data collection for mail-only mode, phone-only mode, and mixed mode conditions will be implemented concurrently for each sample month. Each monthly sample will be fielded over a 6-week data collection period. Data collection for the mode experiment will extend through 2016.
Data collection for each mode will consist of the following:
Mail-only Mode: All OAS CAHPS patients included in the mail-only sample will be sent a first package consisting of a cover letter, the questionnaire, and a pre-addressed, postage-paid return envelope. A second mailing containing a questionnaire and cover letter will be mailed to all sample patients who do not respond to the first mailing.
Telephone-only Mode: In this mode, all sample patients will be contacted via telephone by professional telephone interviewers who will be trained on OAS CAHPS administration procedures, including procedures for working with patients. Telephone interviewers will be trained on the appropriate response to common questions and concerns that patients may have about survey participation. A maximum of 5 telephone contact attempts per patient will be implemented to complete the survey.
Mixed Mode: All sampled patients included in the mixed-mode data collection sample will receive an initial mailing of a questionnaire, cover letter, and postage-paid return envelope that patients included in the mail-only sample will receive. Sample patients assigned to this mode who do not respond to the mail survey will be assigned to the telephone follow-up. Telephone interviewers will make up to 5 attempts to complete the interview by phone with all mail survey non-respondents included in the mixed-mode sample.
The protocol for each mode and the number of weeks for each data collection activity are shown in Exhibit A‑7.
Data from the mode experiment will be analyzed to assess the impact of nonresponse and to determine whether patients rate their dialysis care differently based on data collection mode. Mode experiment data will also be analyzed to determine which, if any, patient characteristics affect patients’ ratings of and reported experience with their outpatient care. CMS will use data from the mode experiment to develop models that will be used to statistically adjust survey results from the national implementation to account for factors that are beyond the control of the outpatient surgery facilities. Results from the mode experiment will not be publicly reported. CMS’s OAS CAHPS contractor will prepare a technical report describing the mode experiment results and recommendations for adjusting survey results for nonresponse, mode, and patient-mix.
Exhibit A-7. Mode Experiment Data Collection Schedule
Mode Experiment |
Schedule |
Mail-only Survey Mail first questionnaire package to all sample members |
Week 1 |
Mail second questionnaire package to all sample members who do not respond to first questionnaire mailing |
Week 4 |
End data collection |
Week 6 |
Phone-only Survey Begin telephone contact with sample members |
Week 1 |
End telephone data collection |
Week 6 |
Mixed-mode Survey Mail questionnaire package to all sample members |
Week 1 |
Initiate telephone follow-up contact for all mail survey nonrespondents |
Week 4 |
Complete data collection |
Week 6 |
Based on data collected for the mode experiment, RTI will conduct analysis to predict responses and ratings as a function of the mode used to administer the survey and, additionally, will estimate the potential effects of patient characteristics outside the control of the ASCs and HOPDs.
The regression models and variable formulations will be empirically driven. Each regression model in the mode experiment analysis will estimate a dependent variable, which will be either one of the responses or a composite of responses. The responses to the survey items of greatest interest usually have ordinal responses. Answer structures are typically yes/no, yes definitely/yes somewhat/no, or a global rating (0, 2,…10). There is no constant linear or ratio relationship inherent in these responses. Two is not necessarily twice as good as 1, and 4 is not twice as good as 2. Such responses may be rigorously modeled using logit methods concerning the probability of selection. However, if the data approximate normality and are not strongly clustered at the extreme ends of the distributions, linear regression with numeric values 1–4 or 0–10 can be used successfully.
Generally, the linear forms of the estimation models will be:
Dependent variable = sum of (coefficients*patient characteristic indicators) + sum of (coefficients*mode indicators) + sum of (coefficients*home health agency indicators).
When there are categorical patient characteristics, such as age groups, one group is omitted from the set of categories included in the model. That group is a reference category to which the effect of related categories is scaled. The facility variables will capture the facility–specific effects in order to isolate the effects of mode and other characteristics.
Each item under consideration for reporting will be tested for inclusion in the appropriate adjustment formula. The outcome of the process will be to determine the mode coefficients to be used as adjusters in the implementation phase and the patient characteristic variables that will be used as adjusters in that phase. The actual coefficients on patient characteristics used in the implementation adjustments will be re-estimated as part of that analysis.
We anticipate requesting data for analysis variables from three sources for the mode experiment analysis: the Facilities (ASCs and HOPDs), the survey itself, and the RTI data collection systems (during the national implementation it will be the survey vendor). Exhibit A-8 contains a list of potential variables for analysis.
Exhibit A-8. List of Potential Variables for Data Analysis
Data Variables |
From ASC or HOPD |
From Survey Respondent |
From Survey System |
Date of birth |
X |
|
|
Gender |
X |
|
|
Medical Record Number |
X |
|
|
Facility type (ASC or HOPD, single-specialty or multi-specialty) |
X |
|
|
Size of facility (patient volume) |
X |
|
|
Ownership type |
X |
|
|
Urban/Rural |
X |
|
|
Type of surgery or procedure |
X |
|
|
Overall health status |
|
X |
|
Mental health status |
|
X |
|
Race and ethnicity |
|
X |
|
Language |
|
X |
|
Education level |
|
X |
|
Did someone help respondent with survey |
|
X |
|
Potential other items on survey |
|
X |
|
Mode of survey administration |
|
|
X |
Number of prompts to elicit response |
|
|
X |
Interview completion date |
|
|
X |
Facility identifier |
|
|
X |
The dependent variables to be analyzed include all survey items (or will be derived from survey items). The independent variables may come from the survey as self-reported characteristics or may have another source (such as enrollment or other administrative data). Because the survey will be fielded to patients associated with all payers, there is no single source of data that can be used for the mode analysis. ASCs and HOPDs will need to provide data from the patient record to their survey vendor (during the national implementation) or to RTI (for agencies participating in the mode experiment). Since we do not know, a priori, which variables will be most important for this population, we may be requesting, for the mode experiment, more than we will ultimately use in our implementation analyses. We recognize the importance of minimizing facility burden, and the mode experiment will be used to reduce the amount of auxiliary data collected in the national implementation.
CMS does not seek this exemption.
There are no exceptions taken to item 19 of OMB Form 83-1.
1 Medicare Payment Advisory Commission’s 2014 Report to Congress on Medicare Payment Policy (MedPac, 2014)
2 Federal Register Volume 79, Number 217 (Monday, November 10, 2014) http://www.gpo.gov/fdsys/pkg/FR-2014-11-10/html/2014-26146.htm
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | cannada |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |