Pilot Report

BUREAU OF PRIMARY HEALTH CARE CLINICAL MEASURES PILOT

FINAL SUMMARY REPORT

REPORTING AND BURDEN

Prepared by

The Johns Hopkins Primary Care Policy Center for Underserved Populations

Project PI: Leiyu Shi, Dr.PH, MBA, MPA

Project Officer: Vanessa Watters, MHA

May 1, 2007

I. BACKGROUND

The Bureau of Primary Health Care (BPHC) Primary Care Data Strategy Pilot Project (hereafter called Pilot) was conducted in early 2007 jointly by the Johns Hopkins Primary Care Policy Center for Underserved Populations and staff from BPHC, to inform BPHC of health centers’ readiness in preparing and reporting selected quality and outcome measures. Specifically, the Pilot was designed to gain insight on health centers’ experiences with clinical performance reporting, provide recommendations to inform the final set of clinical performance measures and data collection methodology, measure the reporting burden for health centers and, if necessary, inform the steps necessary to reduce the reporting burden. The Pilot experience provides guidance to HRSA/BPHC in terms of implementing clinical performance indicators.

The Pilot included a test for selected clinical and outcome measures. These measures are intended to support BPHC’s Primary Care Data Strategy in its effort to improve the program’s ability to demonstrate its impact and effectiveness for patients, payers, and the American public, as well as provide guidance for program improvement. The test was conducted with eight volunteer health centers.

The purpose of this report is to summarize the test methodology, analyze the Pilot sites data results, assess reporting experience and burden, and discuss implications and recommendations.

II. METHODOLOGY

Population and Sample

The targeted population of this Pilot was HRSA funded health centers. The pilot was designed to be conducted within a four to six week time frame. Given the timeline, the targeted population was limited to eight health centers. In order to capture maximum center variations, a purposive sampling method was used which selects sampling elements based on the typical nature of population elements and the purpose of the study. Since this was a data related project, the IT capacity of the center was a critical consideration. In order to capture the variety of centers, we also considered the center’s location (rural versus urban) and affiliation (network versus stand-alone). In addition, we also considered centers that predominantly serve special populations (e.g., homeless only). The following matrix (Figure 1) represents the selection criteria.

Figure 1. Pilot health center selection criteria matrix

LOW INTENSITY OF BURDEN HIGH

	NETWORK	STANDALONE
	EHR/Registry	EHR	Registry	Paper	Special Population
URBAN	1	3	4	6	8
RURAL	2		5	7

We expected the greatest burden to take place in centers with only a paper (i.e., manual) system, followed by registry only (i.e., participation in BPHC healthcare disparities collaborative which requires the use of registry to collect selected measures from selected patients), and the least burden to result in centers with electronic health record (EHR).

Measurement

Five clinical and outcome measures were selected for the Pilot. These were:

1. Childhood Immunization Completion (i.e., percentage of children by 2 years of age with appropriate immunizations),

2. Female Cervical Cancer Screening (i.e., percentage of women 18-64 years of age who received one or more Pap tests within the last two years),

3. Childhood Lead Blood Levels Test (i.e., percentage of patients 9-36 months of age with a blood test for elevated blood lead levels),

4. Adult Hypertension Control (i.e., percentage of adult patients, 18 years and older, with diagnosed hypertension whose blood pressure was less than or equal to 140/90 (adequate control)), and

5. Adult Diabetes HbA1c Levels ¹ (i.e., percentage of adult patients with type 1 or 2 diabetes with most recent hemoglobin A1c (HbA1c) < = 7%, > 7% or < 9%, or > = 9% (poor control)).

Pilot health centers recorded their effort in preparing and reporting these measures and completed a survey on reporting experience. The effort reporting included time spent on such activities as programming, computing, chart abstracting, data inputting, and quality control. The survey (both closed and open ended) covered such topics as ability to report the measures, perceived usefulness of the measures, reliability of the measures, ease in generating the report, system in place to report the measures, sufficiency in labor resources to prepare the measures, and perceived cost benefit in reporting the measures.

Reporting Effort (Burden)

In order to estimate the burden incurred on preparing and reporting the five clinical and outcome measures, we asked the pilot health centers to summarize the time involved in activities related to this task. The activities included programming, computing, chart abstracting, data inputting, and quality control. On average, centers spent 45.28 hours preparing the five measures. Figure 3 illustrates the reporting burden outlined in two tables; by reporting element and by measure.

Figures 3: Breakdown of Reporting Burden
Reporting Element			Measure
programming	14.56 hours		childhood immunization	13.61 hours
chart abstracting	12.56 hours		adult hypertension	9.11 hours
quality control	9.06 hours		female cervical cancer	8.99 hours
data inputting	5.12 hours		adult diabetes	8.38 hours
computing	3.98 hours		childhood lead	5.19 hours

The reporting effort varied widely across pilot health centers. However, contrary to expectation, centers with manual systems were not at a disadvantage in preparing and reporting the measures. In fact, the two centers that spent the most time were an EHR center (27.75 hours per measure) and a registry center (16.6 hours per measure). Instead, the three centers that primarily relied on chart review spent between 3.2 and 7.6 hours per measure.

Reporting Experience

Pilot health centers were also asked to comment on the reporting experience through a closed-ended survey. All Pilot health centers were able to report three of the five measures: female cervical cancer, adult hypertension, and adult diabetes. Six of the eight Pilot health centers were able to report childhood immunization whereas five were able to report childhood lead.

In terms of perceived usefulness of the measures, all Pilot health centers either strongly agreed or agreed that these measures were useful. In terms of perceived reliability of the measures, Pilot health centers either strongly agreed or agreed that most of these measures were reliable. However, for female cervical cancer measure, three centers were not sure of its reliability.

In terms of the easiness of reporting the measures based on the guidance and specifications, centers strongly agreed or agreed that it was easy to report the adult diabetes, adult hypertension, and childhood lead measures. However, some centers found it difficult to report the female cervical cancer and childhood immunization measure.

In terms of whether data collection system was in place to collect these measures, the only measure receiving near unanimous vote was adult diabetes (7 out of 8 centers, with one on the borderline), followed by female cervical cancer (7 out of 8 centers, with one disagreeing), childhood immunization (6 out of 8 centers, with one disagreeing and one borderline), adult hypertension (5 out of 8 centers, with two disagreeing and one unable to report), and childhood lead (5 out of 8 centers, with three unable to report).

In terms of whether there were sufficient labor resources to report the measures, most health centers believed there were sufficient labor resources (5 out of 7 for childhood immunization, 6 out of 8 for female cervical cancer, 7 out of 8 for adult diabetes, 5 out of 7 for adult hypertension, and 5 out of 5 for childhood lead).

Centers were asked to consider the cost implications in report these measures. Most believed it was cost efficient to report these measures (6 out of 7 for childhood immunization, 7 out of 8 for female cervical cancer, 8 out of 8 for adult diabetes, 7 out of 7 for adult hypertension, and 4 out of 5 for childhood lead).

In addition to the close-ended survey, Pilot health centers also shared their experience and opinions through open-ended exit interviews and additional written comments.

IV. DISCUSSION

The Primary Care Data Strategy Pilot Project sponsored by BPHC was conducted to inform BPHC of health centers’ readiness in preparing and reporting selected quality and outcome measures. Lessons learned from the Pilot will be taken into account when institutionalizing quality reporting across all health centers and HRSA funded programs. Overall, the eight Pilot health centers were prepared to report most of the required measures. All Pilot health centers were able to report three of the five measures: female cervical cancer, adult hypertension and adult diabetes. Six of the eight pilot health centers were able to report childhood immunization whereas five were able to report childhood lead. It should be noted that the special population pilot site renders services to an adult population only, and therefore did not report the childhood immunization and child lead measures.

In terms of the burden in preparing and reporting the five clinical and outcome measures, on average, centers spent 45.28 hours preparing the five measures. The measure that demanded the longest time was childhood immunization (13.61 hours), followed by adult hypertension (9.11 hours), female cervical cancer (8.99 hours), adult diabetes (8.38 hours), and childhood lead (5.19 hours). Most of the time was spent on programming and it is anticipated that the programming time associated with these measures will diminish over time as Centers advance from initial start up to establishing routine data extraction methods. Centers that relied on chart reviews did not spend more time preparing and reporting than centers with EHRs or registries.

In terms of reporting experience, although some centers found it difficult to report the female cervical cancer and childhood immunization measures, all Pilot health centers either strongly agreed or agreed that these measures were useful and reliable. Most of the centers believed there were sufficient labor resources to report the measures and that it was cost efficient to report these measures. Centers that report resource constraints were typically smaller centers where there is a perpetual shortage of staff.

Based on these findings, we recommend requesting all five measures to be reported, providing technical assistance to grantees and giving consideration to implementation strategies to help mitigate the reporting burden.

1 The levels indicated for borderline and poor control within the diabetes measure were refined after pilot launch and therefore represent a slight variation from the final version.

Page 4 of 4