General
Feedback
|
Time
to complete:
15
minutes. Less if the language can be clarified/simplified a
little.
5-10
minutes
7
minutes
15
minutes
10
minutes
|
Projected
time to complete the survey will stay the same at 10 minutes or
less
|
How
clear and easy to understand are the survey instructions and
questions?
Consider
removing “postal” unless we are surveying Canadians.
Consider
bullets to make it more digestible and more likely it will be
read. Consider simplifying “other data sources (e.g.,
Medicaid data)” etc.
Generally
easy to understand; but for the question where they asked about
estimating the proportion with disorders, some of the
terminology (elimination disorders, obsessive compulsive and
related disorders, people might not know what the related
disorders are), might need to give more examples.
Easy
Fairly
clear
|
Deleted
"postal" as all locations outside of Continental US
(e.g., Micronesia, Palau, Guam) use ZIP Codes.
Bulleted
the two sentences in Survey Purpose.
Elaborated
elimination disorder by adding examples. Elaborated the
disorders related to obsessive-compulsive disorder by adding
examples.
N/A
N/A
|
Were
there any issues with the flow, formatting, layout, or appearance
of the survey? If so, please explain.
Group
practice questions together and then move on to the "you"
questions.
There
was one question where answers need to add up to 100% that might
be frustrating to complete. Also, if patients have comorbid
disorders, it would add up to more than 100%. Would consider
simplifying this.
No
There
are a series of questions that ask what proportion of patients
were screened, diagnosed, and referred. It took me reading it a
few times before I realized they were different questions. It
would help to italicize or underline those words (screen,
diagnose, refer).
Some
of the response item lists were long. I suggested grouping one
such list into categories.
|
Order
retained to maintain logical flow.
Question
asks about primary/principal diagnosis, not comorbid disorders.
N/A
Bolded
text to differentiate the questions.
Responses
are ordered as related groups/categories.
|
Did
you experience any technology difficulties when accessing and/or
taking the survey? If so, please explain.
No
No
No
|
N/A
|
Please
provide any additional comments or suggestions to improve the
survey (e.g., unnecessary or missing questions or response
options; spelling/grammatical errors; sensitivity issues).
None
other
Overall,
there are A LOT of open-ended % questions. I worry that you are
going to get a lot of missing data, as those kinds of questions
are a bit more stressful for people to answer. Consider a
different way to ask this – give people % categories or a
% slider, maybe?
You
have the definition of a Behavioral Health Disorder at the top
of each page – do you need it on every page, or is it
possible to have it as a footnote, a little “info”
icon, or a call out box on the side? It took me a minute to
realize that I was re-reading something I’d already read
and that it was not a new instruction.
|
N/A
We
have used an input field instead of categories or a slider to
allow for responding with more precision, which is important for
statistical analyses. Also, we have revised to focus on numbers
of patients rather than percentage.
Revised
to present definition of behavioral health disorder on the page
of its first mention.
|
Survey
Question Feedback
|
Please
enter your:
First
Name:
Last
Name:
Primary
Practice Postal or ZIP Code:
Email
address used for receiving communication from [insert PMHCA
program name] program:
Is
it possible to say this more plainly? “Email addresses
will only be used to track survey administration and
completion.” And, why wouldn’t the name do that?
Could you just say “Email addresses are used for internal
tracking only.” or something?
|
Because
some individuals may have multiple email addresses, we request
here the email address that specifically links the respondent
with the PMHCA program that they are associated with for survey
completion tracking. Revised language as recommended.
|
Please
select the primary type of health professional that best
describes you.
You
might consider adding “Child and Adolescent Psychiatry”
like you do for the ped[iatric] subspecialty? This is also a
subspecialty so they may not perceive “other specialist”
as an opportunity to capture that.
This
is a long list – consider grouping them by heading or
something (Physician/advanced practice professional/other…or
something like that).
|
Revised
as recommended.
The
list is presently grouped conceptually without lengthening
further with headings: The physicians are the first 6,
assistants the next 2, nurses the next 4, and behavioral health
practitioners the final 3.
|
Which
PMHCA program services or trainings have you utilized or
participated in during the last 12 months? Select all that apply.
Is
there a natural hierarchy to these services? Could you possibly
simplify this so that it isn’t “utilize or
participated” and “services or trainings”? I’m
not sure the providers will see a difference between utilizing
and participating so maybe get rid of a verb there.
|
Revised
to “used”. Kept “used” and
“participated” per HRSA’s feedback.
|
For
the purpose of this survey, Behavioral Health Disorder is defined
as common, recurrent, and often serious mental health and
substance use disorders that affect an individual’s
behavioral health, defined as the ability to cope with life
stressors, actualize their abilities, learn well, work
productively, and contribute to the community.
Would
this section benefit from definitions that could be found by
hovering over a ? Is there a more straightforward way to say
“actualize their abilities”? Is it necessary to say?
It seems redundant with all the examples that follow in the
sentence.
Is
it possible to make this text into 2 sentences? Is “often
serious” necessary information?
|
Revised
definition.
|
Approximately
how many patients between the ages of 5 and 21 years do you see
per week on average?
Move
this to after the practice questions.
|
Retained
here as a reference for screening, diagnosis, and referral
questions that follow.
|
Which
type of behavioral health screenings does your practice conduct
with patients between the ages of 5 and 21 years?
Ask
both practice questions first and then move on to the “you”
questions.
|
Retained
here to lead in to "you" screening questions.
|
In
the past 30 days, what percentage of the patients you saw between
the ages of 5 and 21 years were screened for behavioral health
disorders (either by you or by other practice staff)? Your best
estimate is fine. By screening, we mean using a question guide or
screening tool (e.g., PHQ-9) intended to identify behavioral
health disorders.
Could
it just say “In the past 30 days, approximately what
percentage…..screened using a question guide or tool by
you or a member of your staff” In this context it is
probably not necessary to specify that the tool is intended to
identify BH disorders. Consider using approximately instead of
the additional complete sentence about best estimate….
|
JBS
and HRSA agreed to use “disorder”.
|
In
the past 30 days, which behavioral health disorders did your
practice screen for among patients (5-21 years)? Select all that
apply.
Is
it necessary to state the ages over and over again?
I
don’t understand why these two skip pattern questions
display differently based on answers to the two screening
questions. For example, if I click general/multidimensional,
then you ask me about screening with the PHQ9 (used as an
example), which the question also defines as a specific
screening. In general, this list is quite long and might benefit
from some cutting and/or grouping.
|
We
have repeated the ages in each question for clear reference for
the respondent.
Removed
examples in the following question to minimize/avoid any
confusion.
|
In
the past 30 days, what percentage of your patients (ages 5-21
years) did you personally diagnose with a behavioral health
disorder? Your best estimate is fine.
Probably
not necessary to keep repeating “personally” over
and over again especially if you group the practice questions
first.
By
this, do you mean an incident diagnosis, or should it include
patients with a prior diagnosis that you’ve seen for a f/u
visit in the past 30 days?
|
“Personally”
retained because some questions are at the individual level,
while some are at the practice level.
Question
solicits the number personally diagnosed in the past 30 days, so
30-day incidence.
|
Of
your patients (ages 5-21 years) that you personally diagnosed
with a behavioral health disorder, what percentage of primary
diagnosis fell into the following categories? Your best estimate
is fine.
Consider:
“Of the patients you diagnosed with a BH d/o in the last
30 days what is the approximate percentage diagnosed with each
of the following.” What do you want them to do if they
diagnosed more than one condition in the same patient since you
require it to add up to 100? You could rephrase: Of all the BH
diagnoses you made in the past 30 days what is the approximate
percentage of each of the following diagnoses? And allow it to
add up to at least 100%.
This
is a little tricky, because you’re requiring that people
take their prior answer % and then translate that into a set of
answers that have to add up to 100. You could either add some
language to the question to make it clear that numbers have to
add up to 100, give an example (e.g., “For example, if you
answered 10% above, the next question requires that you specify
the distribution of diagnoses for that 10% blah blah)
|
The
question focuses on primary/principal, not comorbidities.
Knowing
the number diagnosed is not necessary; respondents need only
identify the percentage diagnosed in each category. The survey
application will validate totals and prompt if < 100%.
|
What
percentage of your behavioral health care referrals for patients
(ages 5-21 years) resulted in a visit to a behavioral health
specialist within 90 days of you referring them?
Have
you defined behavioral health specialist somewhere? Consider
inserting an e.g.
|
Revised
as recommended.
|
Before
participating in the PMHCA program, on average what percentage of
your patients (ages 5-21 years) did you personally screen for
behavioral health disorders? Your best estimate is fine.
During
what period of time? The year before they started participating?
The 30 days before? What are you asking them to average? Per
month? There has to be a way to simplify this. Maybe “In
the year prior to joining the PMHCA program, what percentage of
patients did you screen for BH d/o per month?” or
something. This is a problem throughout this section.
Consider
putting another intro sentence to flag that now we’re
asking you about BEFORE you participated in the program. You’ve
got people answering a lot of % questions and they might be more
focused on that than on reading the nuances of the questions.
|
Period
of time is an estimate of care before participating in PMHCA.
Bolded
the text that highlights that we are asking “before
participation in the program”.
|
How
much has the PMHCA program helped address the unmet behavioral
health care needs of your child and adolescent patients?
Why
unmet? What if it just makes it easier, more efficient, more
effective to meet the bh needs of my patients (still asking
about my individual patients or the practice?) What if there was
a response: there was no unmet need? Here you say child and
adolescent instead of giving the ages. Less painful than the
recurring parenthetical.
|
Revised.
Question now asks how much PMHCA helps to better address
behavioral health care needs. Kept parenthetical for
consistency.
|
Is
there anything else you would like us to know about the impact of
PMHCA program?
I
know it’s an impact study but is it necessary to use that
word in the question? Isn’t anything they write here an
impact? Maybe just “your experience” or “the
effect”? “How else has participating in the PMHCA
program affected your practice?”
|
"Impact"
used for consistency.
|