Crossing The Quality Chasm

Information System Concepts for Quality Measurement Author(s): Brent James Source: Medical Care, Vol. 41, No. 1, Supplement: The Strategic Framework Board's Design for a National Quality Measurement and Reporting System (Jan., 2003), pp. I71-I79 Published by: Lippincott Williams & Wilkins Stable URL: http://www.jstor.org/stable/3767730 Accessed: 12/01/2009 01:59 Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://dv1litvip.jstor.org/action/showPublisher?publisherCode=lww. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit organization founded in 1995 to build trusted digital archives for scholarship. We work with the scholarly community to preserve their work and the materials they rely upon, and to build a common research platform that promotes the discovery and use of these resources. For more information about JSTOR, please contact [email protected].

Lippincott Williams & Wilkins is collaborating with JSTOR to digitize, preserve and extend access to Medical Care.

http://www.jstor.org

MEDICAL CARE Volume41, Number1, Supplement,pp I-71-I-79 02003 LippincottWilliams& Wilkins,Inc.

Information System Concepts for QualityMeasurement BRENTJAMES, MD, MSTAT

Health care information sysBACKGROUND. tems in use today frequently fall short of what is needed to meet the demands for data and reporting on performance. Many observers believe substantial improvements in information systems will be necessary if the potential of a national quality measurement and reporting system (NQMRS) is to be realized. A shared vision will facilitate progress in improving information systems. OBJECTIVES. TO articulate a set of guiding principles and operational steps for the development of functional information systems in health care. RESEARCH DESIGN.Experience in building such systems for one health care delivery system was used to develop an approach. This was discussed with Strategic Framework Board members and integrated with other considerations for going from a local system to one that could accumulate information for na-

tional purposes. FINDINGS.The key elements of a functional information system include provisions that (1) data should be collected once, (2) aggregation of data for higher-level reports should be anticipated, (3) issues related to privacy and confidentiality must be addressed, and (4) measurement systems should include an audit standard. A seven-step process for developing a functional information system is outlined. CONCLUSIONS.A shared national measurement framework is essential because the data systems that health care delivery organizations use are not static. A long-term vision can guide the growth of a data system over time. An NQMRS can be the vehicle that provides the needed vision. Key words: Data collection; functional information system; national quality measurement and reporting system; Strategic Framework Board. (Med Care 2003;41:I-71-I-79)

Health care is inherently an information science. Health care information includes formal knowledge of disease and disease treatment, as well as history and physical examination findings, laboratoryand imaging results, patient preferences and values, and outcomes of health care interventions. The better information a health care professional has, the better he or she can diagnose illness, identify health improvement opportunities, discuss treatment options with patients, implement interventions, and achieve desired outcomes. Similarly,information is necessary for patients to make choices consistent with their values and preferences. Information is also key for planning, managing, and improving the health care

delivery system. Data and information lie at the core of any quality management system.1 In summarizing principles for data collection and management, we looked to well-established models from outside health care, such as banking and transportation safety. In such models, a central agency releases specifications for reportable data. The data are generated by businesses as part of routine operations, then independently reviewed to establish completeness, accuracy, and reliability. Auditors examine the structure and function of the reporting system used within a business. They do not evaluate the results, but only certify that the numbers generated are reasonably correct.

From Intermountain Health Care, Salt Lake City, Utah.

Brent James, MD, MStat, Intermountain Health Care, 36 South State Street, Suite 2100, Salt Lake City, UT 84111-1486. E-mail: [email protected]

Address correspondence and reprint requests to:

1-71

MEDICALCARE

JAMES We envision a similar integrated data system for quality measurement and reporting in health care delivery.Under such a system, health care delivery groups would generate information for internal operations (direct care delivery, management, and improvement) in a way that makes it possible to combine those data into high-level reports for accountability and selection. That will require standards for data collection and reporting and an audit system to ascertain that the data collected are reasonably accurate. A shared national measurement framework is important because the manual and automated data systems that health care delivery organizations use to manage and improve their care are not static. A long-term vision can guide the data system development over time. The Strategic FrameworkBoard (SFB)viewed the national quality measurement and reporting system (NQMRS) as providing that vision so that software vendors and health care delivery groups can develop compatible internal data systems that support shared data for external accountability. Quality measures for specific diseases or clinical support processes are likely to arise from a variety of different organizations and sources. We therefore propose a theoretic framework under which it would be possible to combine measures generated by a range of groups and use the measures together.

Key Elements of a Functional Information System For an information system to be functional, the data contained in it must be accurate, complete, available in a timely manner, and useful for multiple purposes. Few existing information systems in health care have achieved this level of functionality and those that have generally exist within a single organization. For an NQMRS to realize its full potential, the nation will need all health care organizations to have functional information systems. Some key elements of that functionality are highlighted here.

Single-Point Data Collection The SFB recommends single-point data collection when possible because redundancy and the burden of data collection are minimized and data collected at the point of origin are usually more 1-72

accurate and complete than data generated at secondary points. Clear, standard definitions ensure that consistent information is produced over time and across groups, allowing for accurate comparison of inputs and results. This is perhaps easiest to envision in a completely electronic information system environment. For example, when a patient comes in for a visit and has his or her weight and blood pressure measured, the provider would enter those figures directly into the clinical information system (with a hand-held device or perhaps a scale and blood pressure cuff that transmit results directly to the database). The information is collected in a way that allows the information to be used by (1) the physician at that visit (by charting how these values compare to the last few values measured for this patient), (2) the patient in tracking his or her health status over time, and (3) others to evaluate the proportion of the physician's, medical group's, health plan's, region's, state's, and nation's population that is overweight or has high blood pressure.

Combining Data for Multiple Uses The core work of a health care delivery system occurs at the interface between patients and health care professionals. Those interactions generate large volumes of data (eg, a medical record), which in turn drive other data generation and reporting systems supporting health care delivery operations (eg, data for billing, purchasing of supplies, staffing, budgeting, or planning for physical facilities). Single-point, source-level data collection implies that such data will first be useful to manage work processes at the front-line interface, but also can be "rolled up" or aggregated into high-level reports for use throughout a management hierarchy. For example, clinical information generated during a patient encounter can be used for direct patient care as well as for summary reports at the level of individual clinicians, care delivery teams, clinics, regions, delivery systems, or geographic areas. Primary data are usually obtained at the individual patient level. Secondary roll-up data are usually reported at a population level. Although properly structuredprimarydata can nearly always be combined to create secondary measures and reports, data originally collected at a population level often cannot be used to generate individual

Vol.41, No. 1, Supplement patient values because most population-level data are generated from samples (so a value does not exist for every patient) and identifiers that would allow users to link different pieces of information to an individual are often removed to protect confidentiality. Primary data can be aggregated across groups and conditions. Some roll-up reports combine performance measurements across groups of providers or geographic areas within a single clinical measure. Other roll-up reports combine measures across conditions. Identifiable denominators are necessary to create aggregate reports across groups, either by identifiers for individuals or accurate counts of those eligible to be considered in a measure. Common metrics (eg, staging systems) are necessary to create roll-ups across conditions so the groupings are consistent across entities. Risk adjustment must be addressed in both types of roll-up. Aggregate reporting relies on four conceptual underpinnings: (1) classes of outcomes, (2) a common metric, (3) an analytic method (including, when necessary, a risk adjustment system), and (4) patient registries (with master index systems to create accurate denominators). Classes of Outcomes. Nelson and colleagues2 argued that four classes of outcomes data form a balanced "value compass"that generally applies to health care delivery management and reporting: (1) medical, (2) patient functional status, (3) service, and (4) cost. Medical outcomes include complications and achievement of therapeutic goals from the clinician perspective. Patient functional status measures the patient's perspective on treatment effects. Service outcomes include dimensions of the patient-clinician relationship (eg, shared decision-making), access, and convenience. Cost outcomes are the expenditures associated with measured care processes. Three additional classes of measures extend the value compass to the entirety of front-line care delivery. First, patient stratification factors are those that care providers cannot control but that influence outcomes (eg, demographics). Second, referraland treatment indications are diagnosis or treatment factors that are used to determine whether an intervention is appropriate (eg, failure of maximal medical therapy to control angina). Third,key process factors are those that determine outcomes (eg, timely administration of aspirin to heart attack patients). Some useful breakouts within outcomes classes can facilitate identifica-

INFORMATIONSYSTEMCONCEPTS tion of common metrics and lead to consistent analytic methods to combine data in roll-up reports. Common Metrics. Appropriate common metrics for these major outcomes classes must be agreed on. Table 1 lists major outcomes classes with examples of common metrics that allow aggregation. For example, although the indications that determine whether a procedure is appropriate for a particular patient are tailored to a specific clinical scenario, using the appropriateness metric would allow for a report on the proportion of all surgical procedures that are performed for clinically appropriate reasons. Medical outcomes are complex. To create aggregate reports, a classification system may be necessary for grouping clinically diverse outcomes. For example, the set of complications, therapeutic goals, and patient functional status measures one would track for total hip arthroplasty are radically different from those one would track for diabetes mellitus. Having identified a particular clinical condition as an appropriate subject for measurement, a quality measurement system should "fingerprint"the condition in terms of common failure modes (complications or defects), standard therapeutic goals, patient functional status, evidencebased or expert consensus referral and procedure indications, and patient stratification factors. When fingerprinting a particularclinical condition, in addition to preparing a list of common defects, a measures development team should also prepare functional definitions to stage each defect. One approach to summarizing defects across conditions and organizational units is the use of a staging system, such as the one originally suggested by the US Centers for Disease Control and Prevention (Table2). This allows diverse outcomes to be classified on a common metric so that summary performance scores can be reported. Data system design Analytic Methods. should include agreement on the analytic methods that will be used to combine patient data across care delivery groupings and clinical conditions. If the data are being used to make comparisons among entities, the methods will often include severity-of-illness adjustment. Patient Registries. Many useful performance measures take the form of rates (eg, the proportion of diabetic patients who received routine glycosylated hemoglobin tests or the proportion of women screened for breast cancer). Denominator populations are a critical part of any such rate (eg, 1-73

MEDICAL CARE

JAMES TABLE 1. Definitions of Common Metrics for Different Classes of Outcomes Outcome Class

Subclass

of Appropriateness care

Indications

Metric

Descriptive Example of Metric

Defectrate

Patientstreatedeven thoughthey did not meet evidence-basedor professional consensus indications (overuse), and patients who did meet evidence-based or professional consensus indications but did

Defectrate

Key process factors

Medical outcomes

Complications

Therapeuticgoals Patient functional status Service outcomes

Cost outcomes

not receivetreatment(underuse) Patientsfor whom an evidence-basedkey

process step was not performed or where a key process step failed. Defect rate Specific complications Defect rate Failure to achieve treatment goals. Defect rate Failure to achieve functional performance standards; mean shift in performance scores before and after treatment Return and Whether patients preferentially return to the recommend same care provider for future care and recommend that provider to their family, friends and associates Dollars

all diabetic patients in a practice or all women who meet indications for mammography within a defined time period). Patient registries are a means of maintaining the data across a care delivery group or a geographic area so that the denominators needed to generate rates of performance can be generated. Disease registries have a long, positive history within health care delivery measurement and improvement. For example, the American College of

Cost per unit of output (surgical procedure, patient care year)

Surgeons' Commission on Cancer has supported standardized cancer registries in American hospitals since 1922.3 In 1973, the National Cancer Institute coordinated full population data for oncology across five states and four metropolitan areas, and has since expanded its Surveillance, Epidemiology, and End Results program to other pertinent subpopulations.4 Since 1989, the Veterans Administration Health System has maintained a state-of-the-art patient registry for patients with

TABLE 2. Modified Staging System For Complications: Centers for Disease Control and Prevention Stage

Stage 1 Stage 2

Description

Eventoccurred,but patientwas neverat significantriskfor long-termharm.No intervention undertaken. Eventoccurred,but patientwas neverat significantriskfor long-termharm.Interventionundertakento speed recovery or make the patient more comfortable (note that Stage 2 complications consume

Stage 3 Stage4

I-74

extrahealthcareresourcesand so effectcost outcomes). Eventoccurred,patientwas at significantriskfor long-termharm;but an interventionpreventedthat harmfromoccurring(again,with increasedconsumptionof healthcareresourcesand higher costs). The patientsuffereda long-terminjury a. Minorlong-terminjury b. Majorlong-terminjury c. Death

Vol.41, No. 1, Supplement HIV.5 Registries covering heart disease, breast cancer, and many other acute and chronic diseases have provided useful health services information across the entire Swedish population for decades.6-8 Many other disease-specific registries exist. Most rely on voluntary participation of interested individual health professionals or care delivery groups. Measurement systems should specifically identify those measures that require accurate denominator patient populations, define the populations of interest, and recommend methods by which groups treating those populations can reasonably identify, track, and report such patients.

Privacy and Confidentiality of Patient Records The Standards for Privacy of Individually Identifiable Health Information (the Privacy Rulepart of the Health Insurance Portability and Accountability Act of 1996) took effect on April 14, 2001, with compliance by most covered health care entities required by April 14, 2003.9 The Privacy Rule creates national standards to protect individuals' personal health information and gives patients increased access to their medical records. Two issues that might affect the functioning of an NQMRS arise under the new regulation: (1) Use of identifiable patient data. Although an NQMRS would not generate reports on individual patients, it would use individual patient information to evaluate and report the performance of the health care delivery system. The Privacy Rule promotes the use of nonidentifiable patient health data for health services research and lists 19 data elements which would generally make a patient record identifiable unless a competent statistical authority judges otherwise. Those 19 fields include some elements that could be essential to an effective NQMRS, such as date and type of health care service. The most recent revisions to the Rule (August 14, 2002) removed other elements that would be critical to the operation of an NQMRS, such as patient zip code, city, and age, from the proscribed list of patient identifiers (see 164.514(d)2i and Modifications to the Standards for Privacy of Individually Identifiable Health Information-Final Rule; www.hhs.gov/ocr/hipaa/).

INFORMATIONSYSTEMCONCEPTS It may be possible to operate an NQMRS with use of the resultant nonidentifiable Limited Data Sets, in conjunction with research contracts that extend private protections to an NQMRS as required by the Privacy Rule. Even if an NQMRS could not function with nonidentifiable data alone-for example, if an NQMRS needed patient identifiers to link patient records across health care delivery sites-such use of patient data would be covered under the Privacy Rule's health oversight and public health sections (164.512(b) and 164.512(d)). Such an approach would require that the NQMRS be authorized under federal regulation. (2) Patient access to individual NQMRS records. The PrivacyRule gives patients broad access to their own medical records. Maintaining logs to enable patients to review their records and review instances in which their records were referenced could hopelessly burden operation of an NQMRS. However, although an NQMRS would necessarily rely on individual patient information for its internal function, an NQMRS would not use identifiable patient data for individual patient decision-making. Under the Privacy Rule, such uses of patient data are explicitly excluded from patient review (see section 164.524(a)).

Development of Audit Standards The quality measurement and reporting system that we propose rests on local data collection. We anticipate that clinics, hospitals, and care delivery groups will generate data as part of their routine operations, then summarize and report those data to regional, state, and national groups. But any measured result represents the blending of two subcomponents: (1) actual performance and (2) the measurement system. It is often far easier to manipulate the measurement system than to actually deliver excellent performance (in other words, it is easier to look good than to be good). Data system audits attempt to limit the role of the measurement system as a major source of variation in reported results. Measurement system audit standards should focus on completeness and accuracy. Completeness functions at a population level and an individual case level. At a population level, it addresses case finding, or patient registries: are all patients

1-75

JAMES who comprise a denominatorpresent and accountedfor?At an individuallevel, completeness means that all individualdata fields are present and accountedfor.Accuracyassesses the content of each field, directly applying the definitions included in the coding manual to insure that recordcontent accuratelyreflectstrue patientresults. Audit standardsthereforeinclude both a structuralanalysisto assess whethera datasystem has all necessaryfunctionalelements (completeness) and a content analysis to assess whether standardcoding definitionsare consistentlyfollowed (accuracy).Audit standardsthereforeinclude and extend the approachesused to define core data sets. The SFBrecommendsthat each measurement set developed for the NQMRS include explicit standardsfor concurrentdataaudits.Recentexperience with audit fraud in commercial,publicly traded businesses illustrateseveral factors that appear to be criticalwhen implementingaudit systems.First,auditorsmust be completelyindependentfromthe caredeliverygroupstheyreview. In particular,parallelconsultingcontractsappear to offerstrongincentivesfor inappropriatecollusion. Second,auditstandardsrequirea mechanism for regularreview and update based on careful observationof the audit process itself and the ongoingevolutionof the underlyingdatasystems. Care delivery groups that pass independent audit will preparestandardreportsand forward standard data to regional, state, and national agencies. Those groups will analyze and report performanceto the Americanpublic,health oversight agencies,and other interestedparties.

FunctionalSteps in Designinga NationalData System The foregoingdefinitionsand backgroundconceptsprovidethe foundationfora generalmethod to add condition-specificsections to a NQMRS that grows in breadthand depth over time. We discussseven steps to generateindividualmodules withinsuch a system,with sufficientdesigndetail, check points, and transparencyto allow professionals,consumers,and healthoversightgroupsto openlycontributeand criticizeat each point along the way. Our recommendationsare based on models widely used among collaborativeclinical trials groups9'10and related to methods success-

fullyimplementedby severalclinicalimprovement 1-76

MEDICALCARE

collaboratives.The key concept is this: startwith the desired end result, then work backwardto front-linedata collectionand data flow.This approachis more highly structuredthan those applied to quality measurementdesign in some other systems.l It aims to parsimoniouslycollect only those data that lead directlyto usefulreports while comprehensivelycoveringan entireprocess of care. Select High-PriorityClinical Processes The firststep is to focuson high priorityclinical processes.12The conceptof key clinicalprocesses links the internalbusiness of care deliveryto the selection,andmotivation legitimateaccountability, needs of an NQMRS.In one health system,leaders identifiedfour classes of such processes:(1) care and clinical conditions (outpatient/primary clinical care), (2) supportserinpatient/specialty vices, (3) service quality,and (4) administrative supportprocesses. Withininpatientclinicalprocesses,a group of physiciansand nursesfurthergroupedand prioritized conditionsby (1) total patient volume, (2) intensityof care(ie,cost per case),(3) case-to-case variability,and (4) microsystems(teams of cliniThe group cianswho typicallyworktogether).13T14 found that, among more than 600 inpatientconditions,62 accountedfor 92%of all caredelivered on the system'shospitalcampuses.Those62 conditionswere furtherclassifiedinto eightfamiliesof neuromusculoclinicalprocesses (cardiovascular, skeletal,surgical,women and newborn,intensive medicine, intensive pediatrics,intensive behavioral,and oncology). Outpatientclinicalprocesseswere categorized by population group: (1) truly well, (2) latent disease, (3) acute self-limited conditions, (4) chronicconditionsincludingacute exacerbations, and (5) terminalpatients. Fourteenpopulationbased processesaccountedfor more than 90%of all health maintenanceand risk managementactivities for the first two groups of patients (eg, immunization,smokingcessation,and use of seat beltsin automobiles).Fifteenacuteand 15 chronic outpatient conditions accounted for more than 80% of all care deliveredin communitysettings. Manyof the priorityoutpatientclinicalconditions linked directly to priority inpatient clinical conditions. Similaranalyses were undertakenfor clinical supportservices(eg, laboratory,pharmacy,physi-

Vol.41, No. 1, Supplement

SYSTEMCONCEPTS INFORMATION

cal therapy),service quality,and administrative diagramsareflowchartsthattakeadvantageof the natural hierarchyfound in real processes. One supportservices.Forexample,the greatmajorityof a and concerned list begins by generating a simple flowchart of a "caring patients routinely as a key factorin anyhealthcaredelivery clinical process (see Fig. 1, McGlynn15).Next, clinician" those steps in the high-orderflow that hold the experience. The effort requiredto design and implement most potentialfor improvementor most strongly controloutcomes are identifiedand each step is qualitymeasurementand reportingsystems precludes addressing all care processes simulta- expanded as another layer of flowcharts.One continuesuntil a decisionlayeris reached.At that neously. It is therefore imperative that the level,the conceptualflow transformsinto a tradiNQMRSinvest firstin those areasthat will have tional decisionflowchartas is commonlyused in the greatesteffect. practiceguidelinesand protocols. Generate an Explicit Conceptual Model for a Selected Process The second step is to generate and circulate among key decision makersa conceptualmodel for the high-priorityclinicalprocessesselectedin the firststep. Physicians,nurses,technicians,and other front-line care deliverersuse conceptual models to organize and understandtheir work. Suchmodelsprovidea contextforindividualtasks and link them togetherinto a coordinatedworkflow. Those models very often are subconscious, but they are always present-it is impossibleto performcoordinatedworkwithoutthem. Similarly,analysisandreportingrelyon conceptual models.A useful analysisrequiresa conceptual context to link measuresto work processes and outcomes. Just as with front-line improvement work, the conceptualmodels that underlie analysisand reportingoften go unrecognized. The measuresselectionprocessrequiresexplicit conceptualmodels rooted in currentbest understandingof a clinicalconditionand associatedcare deliveryprocesses.15Suchmodels are essentialto: * Obtain consensus for shared improvement work acrosscaredeliveryteams * Prioritizeand focus within complex clinical processes * Provide a context for interpretablemeasurement with shared understandingacross the many groupsthat might use such measures Currentpracticeuses two main forms of conceptual models. The first is hierarchicoutcomes chains.An outcomeschain providesan overview of the entire disease and treatmentprocess, as reflectedin hierarchicintermediateand finalprocess steps and patientoutcomes.The secondform is a conceptualflow diagram.Conceptualflow

Generate a List of Reports and Test Their Utility Conceptual flow diagrams,when combined with the idea of classesof data,providea practical tool for generating balanced sets of outcomes measuresarounda particularclinicalprocess.The approachis simple.A conceptualflow diagramis generatedfor a clinicalprocessdown to the decision level.Eachbox in the layerof the conceptual flowchartimmediatelyabove the decision flowchart is examined while asking the question, "what reports should we routinelygenerate to track performanceand outcomes for this process?"Recordthe resultinglist of reports.Outcomes chainscan be used in a similarway. When the potential reports are identified, model reportsshould be built with use of either realor simulateddata.These shouldbe circulated to those who are expectedto use them for feedback.Is the informationuseful?Is it presentedat the level of analysis needed for the decision maker?How often should the reportsbe generated? This step is frequentlyoverlookedin the design of reportingsystems, and identifyingthe data elements necessaryto generatereportswill save the time and expenseof retoolingto respond to such demandsin the future.

Identifythe Data Elements Necessary to Generate the Desired Reports The fourth step is to use the report list to determinethe data elementsthat will be required to routinely produce informationfor decision makers.A coding manual and self-coding data sheets are then designed to obtain the data elements. 1-77

MEDICALCARE

JAMES Coding Manuals. Coding manuals list every data element in a proposed measurement set, with (1) functional definitions, (2) complete descriptions of coding schemes, (3) instructions regarding missing and unknown data, and (4) descriptions of primary sources for each data element. Self-coding Data Sheets. Self-coding data sheets reduce a coding manual to a manual data entry sheet on which all data elements are shown as labeled data boxes. All definitions, coding schemes, data sources, and other related instructions necessary for accurate, complete data entry, are included as part of the sheet. Self-coding data sheets are shorthand coding manuals for practical data entry use and are potent tools for designing succinct data systems that function well. This scheme can be pilot-tested to determine how the necessary data collection can be integrated into the flow of patient care without hampering care delivery operations, and whether the resulting reports can be produced in a timely manner.

Plan the Flow of Data at the Level of Care Delivery The fifth step is to identify which data are already automated and which must be obtained in the course of care delivery. Converting handcoded data sheets into automated data acquisition systems can substantially enhance the efficiency of the information system. The data acquisition strategy then must be designed, with those on the front line providing input into how actual data capture, recording, and reporting will be accomplished. This may require some negotiation (what we want versus what we can get). This step can also generate a plan for data system improvements that can be introduced over time as the data system goes through its regular upgrade cycle. Data flow planning also addresses methods to combine, preprocess, store, and report information across different care delivery units.

generate all the reports originally identified. Some data elements may have accidentally fallen off or new data elements may be required as a result of redesign on the front lines.

Implement the Measurement and Reporting System The last step is to implement the system. Information system design rests on careful attention to these steps, or their equivalents. Experience in real data systems has shown that shortcuts during the planning and testing phase are likely to lead to increased cost and decreased functionality of the final product.

Conclusions An efficient NQMRS should simultaneously address measurement for accountability/selection and measurement for improvement.1 Many others have addressed the use of data systems for quality measurement,16-20but the key concept we propose is that of careful data system design for the dual purposes of local operations and national reporting. Properly designed, a data system built to support front-line clinical process management and improvement can also produce data for summary reports that support accountability,selection, and motivation. Such systems can minimize the burden of data collection and reporting that is often associated with quality measurement activities and ensure that the data are accurate and complete, available when needed, and useful to decision makers. We identified key elements and concepts relating to the structure and function of effective data systems. On that foundation, we propose a series of concrete steps that all proposed clinical quality measures included in the NQMRS should follow.

References 1. Berwick DM, James B, Coye MJ. The connec-

Test the Final Reporting System The sixth step is to test the resulting final reporting system before full-scale implementation is attempted. This iteration is necessary because it is likely that compromises were made in the previous step that affect the ability of the system to

1-78

tions between quality measurement and improvement. Med Care 2002;41(suppl):I-30-I-38.

2. Nelson EC, Mohr JJ, Batalden PB, et al. Improving health care, part 1: the clinical value compass. Jt Comm J Qual Improv; 1996;22:243-258.

3. American College of Surgeons. Availableat: http://www.facs.org/dept/cancer/coc/. Accessed August 12, 2002.

Vol. 41, No. 1, Supplement 4. National Cancer Institute. Available at: http:// www.seer.cancer.gov/about/. Accessed August 12, 2002. 5. Rabeneck L, Menke T., Simberkoff MS, et al. Using the national registry of HIV-infected veterans in research: lessons for the development of disease registries. J Clin Epidemiol 2001;54:1195-1203. 6. Hemminki K, Granstrom C, Czene K. Attributable risks for familial breast cancer by proband status and morphology: a nationwide epidemiologic study from Sweden. Int J Cancer 2002;100:214-219. 7. Hammar N, Alfredsson L, Rosen M, et al. A national record linkage to study acute myocardial infarction incidence and case fatality in Sweden. Int J Epidemiol 2001;30(suppl 1):S30-S34. 8. Stenestrand U, Wallentin L. Early revascularisation and 1-year survival in 14-day survivors of acute myocardial infarction: a prospective cohort study. Lancet 2002;359:1805-1811. 9. Department of Health and Human Services. Available at: http://www.hhs.gov/ocr/hipaa. Accessed August 12, 2002. 10. Pocock SJ. Clinical Trials:A PracticalApproach. New York,NY: John Wiley & Sons; 1983. 11. Halpern J. The measurement of quality of care in the Veterans Health Administration. Med Care 1996;34:MS55-MS68. 12. Institute of Medicine Committee on Quality of Health Care in America. The value of organizing

INFORMATION SYSTEMCONCEPTS around priority conditions. In: Corrigan JM, Donaldson MS, Kohn LT,eds. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001, p. 98-110. 13. Batalden PB, Mohr JJ, Nelson EC, et al. Continually improving the health and value of health care for a population of patients: the panel management process. Q Manage Health Care 1997;5:41-51. 14. Nelson EC, Batalden PB, Mohr JJ, et al. Building a quality future. Front Health Serv Manage 1998;15:3-32. 15. McGlynn E. Selecting common measures of quality and system performance. Med Care 2002; 41(suppl):I-39-I-48. 16. Nelson EC, Splaine ME, Batalden PB, et al. Building measurement and data collection into medical practice. Ann Int Med 1998;128:460-466. 17. Nelson EC, Batalden PB. Patient-based quality measurement systems. Q Manage Health Care 1993;2:18-30. 18. Iezzoni LI. Assessing quality using administrative data. Ann Int Med 1997;127(suppl):666-674. 19. McDonald CJ, Overhage JM, Dexter P, et al. A frameworkfor capturingclinical data sets from computerized sources. Ann Int Med 1997;127(suppl):675-682. 20. Palmer RH. Process-based measures of quality: the need for detailed clinical data in large health care databases. Ann Int Med 1997;127(suppl):733-738.

1-79