CCBHC Demonstration Evaluation Clinic Interview Protocol
Unless otherwise noted, questions will be asked of all cohorts included in interviews in a given year.
Obtain verbal consent to record
We would like to learn about your current role(s)/position(s). What are your key responsibilities related to the CCBHC demonstration?
For how long have you held this position?
How has demonstration implementation been going for your CCBHC? What has been your clinic’s greatest success and what has been your greatest challenge during the past year?
[BSCA cohort 1] What features of the CCBHC model have worked well so far during the implementation process? How have these improved work/processes in your clinic?
What has worked well for clients?
What has worked well for clinic staff?
[BSCA cohort 1] Has your organization changed since the demonstration began in your state? If so, what are the biggest ways your organization has changed?
Did you have to make these changes to meet the certification requirements?
How has the state supported your implementation of the model over the last year?
Do you feel like you have received enough support from the state for implementing the CCBHC model? Why or why not?
Do you participate in any learning collaboratives or peer learning/TA opportunities? Have you found them useful?
We would like to hear about your CCBHC’s experience with demonstration requirements related to improving access to care.
What activities did your clinic implement to increase access to care in response to the findings from the community needs assessment that was conducted when the state applied for the demonstration?
Did you implement any activities specifically to increase access to care among certain target populations?
Have there been activities that you planned to implement but were unable to?
Why were you unable to implement these activities?
The CCBHC criteria specify the timeframes that triage, assessments, and first clinical appointments must occur within. Has your CCBHC experienced any challenges in meeting these timeframes?
Have you heard any feedback from people receiving services regarding the timeliness of these initial contacts?
How do these timeframes compare to what your clinic was able to offer before the demonstration?
What changes did your clinic need to make to meet this component of the criteria?
What hours/days are CCBHC services available for clients at your CCBHC, in person? Are all nine types of CCBHC services available at the same times, or are certain services only available at specific times of day/days of the week?
What happens when clients seek routine or urgent-care appointments outside of regular business hours (e.g., weekends / evenings)? How do they get care in these scenarios?
Is medical record information for care and advice after hours integrated with business hours records or systematically shared with daytime staff?
How accessible is the CCBHC by public transportation?
Do you offer transportation support to help clients get to care, such as shuttle service, Medicaid cab?
Are you carrying out any outreach activities to reach people who have never been seen by your CCBHC who are not engaged in services?
What kinds of outreach are you conducting (e.g., developing relationships with community organizations, advertising, etc.)?
How does this outreach differ from what you were doing prior to joining the demonstration? (e.g., focusing on specific target population(s))
What about any outreach activities to reach people who have been seen by your CCBHC before but who are not currently engaged in services?
How does this outreach differ from what you were doing prior to joining the demonstration? (e.g., focusing on specific target population(s))
What activities has your clinic engaged in to meet the needs of children and families?
Are these activities new since your clinic became a CCBHC?
What has worked well as you have implemented these activities? What could improve your ability to serve children and families?
What information and/or services are available to CCBHC clients through a secure electronic system? (e.g., health information, clinical visit summaries, 2-way communication with the practice, emails to notify clients about needs.)
Does the CCBHC have an interactive website or patient portal to support CCBHC client access?
Does your CCBHC offer services via telehealth? If yes:
What technologies are used?
Are telehealth services offered to all populations and for all services or only specific populations or services?
Are telehealth services available to children and youth?
Are telehealth services available to people with disabilities (other than behavioral health disabilities)?
Are any text messaging or mobile applications being used to reach clients or improve clients’ access to CCBHC services?
Has your CCBHC been able to meet the needs of people with limited English proficiency or language-related disabilities? For example, does the CCBHC offer off-site interpreter and translation services?
We would like to hear about your CCBHC’s experience with staffing requirements of the CCBHC model.
Please describe the structure of your CCBHC management team.
How many non-clinical and clinical staff are part of your management team?
Has your clinic been successful in hiring the numbers and types of staff reflected in the CCBHC’s staffing plan?
Was your staffing plan informed by your clinic’s community needs assessment?
Have certain types of staff been more challenging to hire than others? Has this changed since the beginning of the demonstration?
What factors have influenced your clinic’s ability to hire required staff?
Does the staff composition of your CCBHC include the following? Why or why not?
Probe on:
Medically trained behavioral health care providers who can prescribe and manage medications independently under state law, including medications used to treat opioid, alcohol, and tobacco use disorders
Licensed or certified substance use treatment counselors or specialists and addiction medicine physicians or specialists
Providers with expertise in addressing trauma and promoting the recovery of children and adolescents with serious emotional disturbance and adults with serious mental illness
Did your CCBHC have these types of staff before becoming a CCBHC?
[CARES Act Cohort only] Has your clinic been successful in retaining staff needed to implement the CCBHC model during the demonstration?
[If yes]: What has helped your clinic to retain staff?
[If no]: What challenges has your clinic faced in retaining the required staff? For example, have certain staff types been more difficult to retain than others?
What caused these challenges?
How has your clinic and the state addressed these challenges?
Has the clinic’s ability to retain staff differed at different points during the demonstration? What accounts for this variation if so?
What type of cultural competencies do you seek in CCBHC service providers?
Do staff receive cultural competency training to ensure that they can meet the needs of the population served? If yes, please describe:
Does your clinic monitor staff to ensure they are providing culturally competent services? How if so?
Does the cultural and racial composition of clinic staff reflect that of people served by the clinic?
Have you faced any challenges in working to ensure the cultural and racial composition of clinic staff reflects people served by the clinic?
Has your CCBHC experienced any other challenges associated with maintaining staff or training requirements, consistent with the certification criteria?
How do you monitor the performance of your CCBHC? [Open-ended, then prompt with the following:]
What sources of data do you use?
Do you use any of the data you collect for the CCBHC in any additional ways? How? And how often? (e.g., quarterly, bi-annual or annual presentations to the team?)
In what way are the data analyzed? What are plans for ongoing/future data analysis?
How does your CCBHC use the quality measures required for the demonstration?
What, if any, are some of the challenges you have encountered in using the measures to improve care?
What would help you be able to use the quality measures more effectively?
Could you reflect on your experience collecting and reporting the CCBHC quality measures?
What are some of the challenges you have encountered?
How have the state or other groups supported you, if at all?
Are you able to calculate the quality measures for the specified populations? If so, have you experienced any difficulties?
[CARES Act cohort] Are the required quality measures / [BSCA cohort 1] Do you expect the required quality measures will be: helpful for measuring and improving the quality of your CCBHC and the quality of the care that your CCBHC provides?
[CARES Act cohort] Does your state share performance on the demonstration quality measures with you?
What specifically does your state share? For example, does it share your clinic’s performance only or does it share performance of other CCBHCs in your state?
How frequently does the state share quality measure performance with your clinic?
Have you made any changes to the way your clinic implements the model as a result of the information provided by the state?
Reflecting on the CCBHC model overall, do you think it improves the quality of behavioral health care (compared to standard practice)?
If yes: Which aspects of the model improve care quality the most?
Has quality improved for all populations or only some?
How has the quality of care changed over time?
Are there any CCBHC requirements that do not significantly improve the quality of care? Why? (For example, because they are too burdensome, not well-received, etc.)
How, if at all, have your CCBHCs’ EHR/HIT systems helped your clinic to deliver higher-quality care? What have been some of the challenges in using your EHR/HIT systems for this work?
Is your clinic part of a health information exchange in your state? Has this been helpful if so?
Are you alerted if your client is seen by another provider and how?
[CARES Act cohort (and BSCA Cohort 1 if not asked above) What information and/or services are available to CCBHC clients through a secure electronic system/patient portal? (e.g., health information, clinical visit summaries, treatment plan, two-way communication with the practice, emails to notify clients about needs)? Does the CCBHC have an interactive website or patient portal to support CCBHC client access?
To what extent did you have these technologies and workflows in place prior to the demonstration? Which were added because of the demonstration?
[CARES Act cohort] Has your clinic been eligible to receive quality bonus payments for meeting certain quality measure thresholds under the demonstration?
[If yes] Have you found that the quality bonus payment system offers an incentive to provide improved quality care? Why or why not?
What specifically is it about the quality bonus payment system that does or does not incentivize higher quality? For example, is it the amount of the payment? The measures the state is using? The ability to benchmark your clinic to others in the state?
[BSCA cohort 1] Do you expect your clinic will be eligible to receive quality bonus payments for meeting certain quality measure thresholds under the demonstration?
[If yes] Do you think that the quality bonus payment system offers an incentive to provide improved quality care? Why or why not?
What specifically is it about the quality bonus payment system that does or does not incentivize higher quality? For example, is it the amount of the payment? The measures the state is using? The ability to benchmark your clinic to others in the state?
Have you encountered any difficulties reporting costs via the cost reports?
What services and costs does your CCBHC provide to Medicaid beneficiaries that are not allowed to be included in the cost report?
How does the clinic monitor reporting of PPS claims and encounter data?
How often are records checked for accuracy and by whom?
What costs does your CCBHC have that are not covered by the PPS?
[Interviewer note: If needed, probe on costs for people who are not eligible for Medicaid]
How do you assess whether your clinic is meeting the needs of people receiving services, including children, adolescents, and families?
Does your clinic solicit feedback from people receiving services, both formally and informally? For example, does the CCBHC collect data related to satisfaction beyond the required quality measures?
What information is collected?
How does your clinic use the information collected?
[If not addressed in response to question 2] Do you solicit feedback from children receiving services and/or their families?
What information is collected?
How does your clinic use the information collected?
Has any of the feedback from people receiving services from your clinic been unexpected or surprising?
Which of your clinics’ activities to improve access and quality have clients responded most favorably to? Which have clients not taken advantage of?
How, if at all, has the clinic integrated people with lived experience and their family members into clinic leadership and decision-making?
Has your state shared whether it plans to sustain the CCBHC model beyond the end of the demonstration?
Does your CCBHC plan to remain a CCBHC through another funding mechanism (e.g., SAMHSA grant, state plan amendment) after the demonstration ends?
Would you continue to provide all of the services you are providing under the demonstration and retain all types of staff? Why?
Do you have any concerns regarding CCBHC program sustainability?
Obtain verbal consent to record
We would like to learn about your current role(s)/position(s). What are your key responsibilities related to the CCBHC demonstration?
For how long have you held this position?
How has demonstration implementation been going for your CCBHC? What has been your greatest success and what has been your greatest challenge during the past year?
[BSCA cohort 2] What features of the CCBHC model have worked well so far during the implementation process? How have these improved work/processes in your clinic?
[BSCA cohort 2] How, if at all, has your organization changed since your CCBHC began implementing the demonstration?
How has the state supported your implementation of the model over the last year?
Do you feel like you have received enough support from the state for implementing the CCBHC model? Why or why not?
Do you participate in any learning collaboratives or peer learning/TA opportunities? Have you found them useful?
Does your CCBHC also have a CCBHC Expansion grant from SAMHSA?
How are the programs overlapping or complementing one another at your CCBHC? For example, does your CCBHC use the Expansion grant to serve populations not covered under the Medicaid PPS?
Have you used any technology to extend your clinic’s reach?
Does your CCBHC provide services in locations outside of the physical clinic? In what other locations, if any, are CCBHC services offered? How did you select these other locations?
Did you offer care in these other locations prior to the demonstration?
What has worked well about offering services in locations outside of the physical clinic?
What policies or procedures could improve your CCBHC’s ability to effectively serve people in locations outside of the physical clinic?
What activities has your clinic engaged in to meet the needs of children and families?
Are these activities new since your clinic became a CCBHC?
What has worked well as you have implemented these activities? What could improve your ability to serve children and families?
How do your CCBHC’s activities to increase access compare to non-CCBHC behavioral health clinics in your area?
What does your CCBHC do differently than non-CCBHC clinics in terms of efforts to ensure access to care?
Which of your clinics’ activities to improve access have clients responded most favorably to? Which have clients not taken advantage of?
We would like to hear about your CCBHC’s experience with demonstration requirements related to improving access to care.
What access-related activities did your clinic implement in response to the findings from the community needs assessment that was conducted when the state applied for the demonstration?
Have there been activities that you planned to implement but were unable to?
Why were you unable to implement these activities?
The CCBHC criteria specify the timeframes that triage, assessments, and first clinical appointments must occur within. Has your CCBHC experienced any challenges in meeting these timeframes?
Have you heard any feedback from people receiving services regarding the timeliness of these initial contacts?
How do these timeframes compare to what your clinic was able to offer before the demonstration?
What changes did your clinic need to make to meet this component of the criteria?
What hours/days are CCBHC services available for clients at your CCBHC, in person? Are all nine types of CCBHC services available at the same times, or are certain services only available at specific times of day/days of the week?
What happens if clients seek routine or urgent-care appointments outside regular business hours (e.g., weekends / evenings)?
Is medical record information for care and advice after hours integrated with business hours records [or systematically shared with daytime staff]?
How accessible is the CCBHC by public transportation?
Do you offer support to clients in accessing clinic, such as shuttle service, Medicaid cab?
Are you carrying out any outreach activities to reach clients who are not engaged in services?
What kinds of outreach are you conducting (e.g., developing relationships with community organizations, advertising, etc.)?
How does this outreach differ from what you were doing prior to joining the demonstration? (e.g., focusing on specific target population(s))
What information and/or services are available to CCBHC clients through a secure electronic system? (e.g., health information, clinical visit summaries, 2-way communication with the practice, emails to notify clients about needs.) Does the CCBHC have an interactive website or patient portal to support CCBHC client access?
Does your CCBHC offer services via telehealth? If yes:
What technologies are used?
Are telehealth services offered to all populations and for all services or only specific populations or services?
Are any text messaging or mobile applications being used to reach clients or improve clients’ access to CCBHC services?
Has your CCBHC been able to meet the needs of people with limited English proficiency or language-related disabilities? For example, does the CCBHC offer off-site interpreter and translation services?
What services and evidence-based practices did your clinic add to meet certification requirements?
Were any services particularly challenging to add initially? Why?
What strategies helped you to overcome these challenges? Did the state provide any support or guidance?
How does your clinic monitor fidelity in the delivery of evidence-based practices?
How do you feel about the service requirements that the state has established?
Are they applicable to your clinic’s client populations?
Do the requirements include any services that your clients are not using?
What small or big changes to the scope of services required by your state would you find helpful?
How is the provision of evidence-based practices going?
Has your CCBHC had any trouble providing 24/7 access to mobile crisis care?
[If yes] What approaches has your clinic used to address these challenges? Have you ultimately been able to put 24-hour coverage in place?
[BSCA Cohort 2] What services and evidence-based practices did your clinic add to meet certification requirements?
Were any services particularly challenging to add initially? Why?
What strategies helped you to overcome these challenges? Did the state provide any support or guidance?
Has your clinic been able to maintain the required CCBHC services over time?
Have some services been more difficult to maintain than others?
What has helped you maintain the required services?
What challenges have you encountered in providing particular types of services?
How have you tried to overcome these challenges?
Has your CCBHC encountered any challenges in providing services to certain target populations?
Does your CCBHC have a Designated Collaborating Organization (DCO) or multiple DCOs?
[Ask remaining questions in this section only of clinics with DCOs]
Describe the services that your DCO(s) provide(s) for CCBHC clients, and reasons why they are provided by a DCO rather than provided in CCBHC.
If multiple DCOs, which DCOs provide which services?
Please describe the relationship between the CCBHC and DCO(s) (if applicable):
Are relationships with DCO(s) new (since CCBHC certification), pre-existing informal, or pre-existing formal (i.e., prior contractual agreement with DCO)?
What is the distance between CCBHC and DCO(s)?
How has your CCBHC formalized its agreement with the DCO. That is, does your CCBHC have a memorandum of understanding or contract with the DCO?
What has been most challenging or successful about these relationships with DCOs?
How do clients get from one location to another?
Does the program CCBHC assist with transportation between locations [note: transportation of clinic users is not an allowed cost under the CCBHC demonstration]?
How many clients have been referred to DCO? How are they identified?
What steps does your CCBHC take to ensure that required CCBHC services provided by DCOs are delivered in a manner that meets the standards set in the CCBHC certification criteria?
Describe the process for sharing clinical information and other data between CCBHCs and DCOs.
How is the information shared?
Are electronic records shared between CCBHC and DCO providers?
What reporting requirements are in place between your CCBHC and DCO(s)?
Do DCOs report encounter data?
Do DCOs report information required to calculate required quality measures?
Have you had any challenges in managing payment for care provided in DCOs?
How is client care coordinated for people receiving services from your CCBHC (i.e., designated care manager, case manager, care coordinator; direct communication between providers)?
What types of staff are involved in care management?
What types of staff are involved in person- and family-centered planning?
What is done to manage CCBHC clients’ medications across different prescribers?
What is the average CCBHC client caseload for full-time care managers and other providers?
How, if at all, has the care coordination your CCBHC provides changed as a result of joining the demonstration?
Could you briefly describe the types of coordination activities your CCBHC engages in with external partners and provider organizations?
Please describe the process for referrals to external healthcare services. How is your referral process to other healthcare providers going? What about to non-healthcare providers or supports?
What is the process for receiving referrals to the CCBHC (e.g., from crisis center, hospital, etc.)?
How are referrals to external services (e.g., specialists) managed? Are Care Managers involved, or other staff?
How are referrals tracked, with follow-up? [e.g., paper or electronically, sharing clinical information, tracking status of referrals, following up to obtain specialist reports] How often?
Which staff are responsible for follow-up with referrals?
How does your CCBHC learn about care transitions among the people you serve? For example, how does your CCBHC learn if someone you serve has a hospital stay or emergency department visit?
How about internal coordination activities? How does your clinic staff collaborate to monitor and address clients’ behavioral health conditions and their treatment?
How do providers share information with one another?
Who is responsible for coordinating clients’ care?
Does your CCBHC have partnerships with other organizations that are not formally designated as DCOs? If yes:
How are those relationships going? What has been most challenging about these relationships? What about most successful?
Has it been difficult to obtain or retain formal, signed agreements with certain types of partners?
Is health IT supporting your CCBHC’s partnerships with other providers? How if so?
How do you monitor the performance of your CCBHC? [Open-ended, then prompt with the following:]
What sources of data do you use?
Do you track CCBHC clients’ utilization information related to health care costs? (e.g., emergency room visits, hospital admissions, generic vs. prescription medications)?
Do you use any of the data you collect for the CCBHC in any additional ways? How? And how often? (e.g., quarterly, bi-annual or annual presentations to the team?)
In what way are the data analyzed? What are plans for ongoing/future data analysis?
Who is responsible for data analysis (e.g., internal staff member, contracted external evaluator, etc.)?
Could you reflect on your experience collecting and reporting the CCBHC quality measures?
How are quality measures calculated and reported?
What are some of the challenges you have encountered?
How have the state or other groups supported you, if at all?
Are you able to calculate the quality measures for the entire CCBHC population? If so, have you experienced any difficulties?
[BSCA Cohort 1] Are the required quality measures / [BSCA Cohort 2] Do you expect the required quality measures will be: helpful for measuring and improving the quality of your CCBHC and the quality of the care that your CCBHC provides?
Why or why not?
Does your CCBHC collect or plan to collect and use the optional measures? What about additional measures not included as required or optional for CCBHCs? Why or why not?
[BSCA Cohort 1] Does your state share performance on the demonstration quality measures with you?
What specifically does your state share? For example, does it share your clinic’s performance only or does it share performance of other CCBHCs in your state?
How frequently does the state share quality measure performance with your clinic?
Have you made any changes to the way your clinic implements the model as a result of the information provided by the state?
Did your clinic provide any input to the state about which PPS option to select?
If yes: What input did your clinic provide? Why was the input your clinic provided important to your clinic?
How does your experience being paid under a PPS compare to your experience being paid before the demonstration or at other organizations you have worked for?
What proportion of your billing work involves the PPS as opposed to other payers?
Is the administrative burden of submitting claims different for people covered by the PPS system? In what ways?
How is the payment system working for your clinic? Have you encountered any difficulties or received any feedback from staff or clients?
How accurate or fair do you think the PPS rates are for your clinic?
To what extent do the payment rates cover the costs of people you serve who have Medicaid?
Have the requirements to operate as a CCBHC increased the costs of covering other clients, such as the uninsured? How do you cover these increased costs if so?
[If PPS-2 system]: How are the various components of the rate mechanism working at your clinic (i.e., stratification of rates by patient severity, outlier payments, and quality bonus payments)?
Have you encountered any issues regarding payment for certain types of people? For example, those who are dually eligible/enrolled (Medicare/Medicaid)? For people who are recipients of 1915(c) waivers?
How does the clinic handle billing if a client receives services from more than one DCO in a single day?
How are people whose care is covered by the PPS distinguished from people not covered by the PPS?
Are people with Medicaid tracked separately from others?
Are services for people covered by the PPS managed differently than the way services are managed for other people?
How, if at all, has the PPS supported your clinic’s efforts to improve access, quality of care, and scope of services?
What do you think accounts for the PPS’ impacts? For example, are they from net increases in reimbursement, reductions in administrative burden, or coverage of previously uncovered services?
Has the PPS facilitated any specific innovations? For example, innovative ways of service provision, use of technology, new types of staff?
Have you encountered any difficulties reporting costs via the cost reports?
What costs does your CCBHC have that are not covered by the PPS?
[Interviewer note: If needed, probe on costs for people who are not eligible for Medicaid]
[BSCA Cohort 1] Has your clinic been eligible to receive quality bonus payments for meeting certain quality measure thresholds under the demonstration?
[If yes] Have you found that the quality bonus payment system offers an incentive to provide improved quality care? Why or why not?
What specifically is it about the quality bonus payment system that does or does not incentivize higher quality? For example, is it the amount of the payment? The measures the state is using? The ability to benchmark your clinic to others in the state?
[BSCA cohort 2] Do you expect your clinic will be eligible to receive quality bonus payments for meeting certain quality measure thresholds under the demonstration?
[If yes] Do you think that the quality bonus payment system offers an incentive to provide improved quality care? Why or why not?
What specifically is it about the quality bonus payment system that does or does not incentivize higher quality? For example, is it the amount of the payment? The measures the state is using? The ability to benchmark your clinic to others in the state?
Has your state shared whether it plans to sustain the CCBHC model beyond the end of the demonstration?
Does your CCBHC plan to remain a CCBHC through another funding mechanism (e.g., SAMHSA grant, state plan amendment) after the demonstration ends?
Would you continue to provide all of the services you are providing under the demonstration and retain all types of staff? Why?
Do you have any concerns regarding CCBHC program sustainability?
Mathematica®
Inc.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | 1-column report template |
| Author | Allison Wishon |
| File Modified | 0000-00-00 |
| File Created | 2024-12-10 |