Number of patients is adjusted based on subspecialty and the type of patient (new, follow-up, or surgical). The numbers on the x-axis are for ordering purposes only and do not refer to specific practices.
Total value of work determined by Medicare relative value units vs actual practice costs. Total value of work divided by practice cost gives the adjusted cost used in the efficiency index. The line indicates a ratio of 1.
Values for quality of process are measured using the quality questionnaire instrument. The numbers on the x-axis are for ordering purposes only and do not refer to specific practices.
Calculated efficiency index for 36 practices. The numbers on the x-axis are for ordering purposes only and do not refer to specific practices.
eEquation. Medicare Value of Work
eAppendix. Quality Checklists
eTable 1. Patient Weighting Used to Calculate the Adjusted Number of Patients
eTable 2. CMS-Assigned RVUs for the 2012 Fiscal Year for Each Subspecialty
eTable 3. Example Calculations for the Efficiency Index
Chen A, Kim EA, Aigner DJ, Afifi A, Caprioli J. Index to Estimate the Efficiency of an Ophthalmic Practice. JAMA Ophthalmol. 2015;133(8):924-929. doi:10.1001/jamaophthalmol.2015.1447
A metric of efficiency, a function of the ratio of quality to cost per patient, will allow the health care system to better measure the impact of specific reforms and compare the effectiveness of each.
To develop and evaluate an efficiency index that estimates the performance of an ophthalmologist’s practice as a function of cost, number of patients receiving care, and quality of care.
Design, Setting, and Participants
Retrospective review of 36 ophthalmology subspecialty practices from October 2011 to September 2012 at a university-based eye institute.
The efficiency index (E) was defined as a function of adjusted number of patients (Na), total practice adjusted costs (Ca), and a preliminary measure of quality (Q). Constant b limits E between 0 and 1. Constant y modifies the influence of Q on E. Relative value units and geographic cost indices determined by the Centers for Medicare and Medicaid for 2012 were used to calculate adjusted costs. The efficiency index is expressed as the following: E = b(Na/Ca)Qy. Independent, masked auditors reviewed 20 random patient medical records for each practice and filled out 3 questionnaires to obtain a process-based quality measure.
Main Outcomes and Measures
The adjusted number of patients, adjusted costs, quality, and efficiency index were calculated for 36 ophthalmology subspecialties.
The median adjusted number of patients was 5516 (interquartile range, 3450-11 863), the median adjusted cost was 1.34 (interquartile range, 0.99-1.96), the median quality was 0.89 (interquartile range, 0.79-0.91), and the median value of the efficiency index was 0.26 (interquartile range, 0.08-0.42).
Conclusions and Relevance
The described efficiency index is a metric that provides a broad overview of performance for a variety of ophthalmology specialties as estimated by resources used and a preliminary measure of quality of care provided. The results of the efficiency index could be used in future investigations to determine its sensitivity to detect the impact of interventions on a practice such as training modules or practice restructuring.
In 2011, US health care spending accounted for 17.3% of the gross domestic product, or $8175 per capita, which is far greater than any other country in the world.1 In 2014, US health care spending was projected to grow by 8.4% largely due to the increase in coverage from the Patient Protection and Affordable Care Act passed in 2010.2 However, this high rate of spending does not provide better outcomes for patients as compared with data from other countries.3 The disparity between spending and quality of care points to reducible waste inherent to the current health care system. Components of the Patient Protection and Affordable Care Act aim to address some of these deficiencies while expanding care to 48 million uninsured individuals.4 To expand coverage while reducing expenditure, methods to compare cost of resources with quality of care will need to be developed and tested.5 A metric of efficiency, a function of the ratio of quality to cost per patient, will allow the system to better measure the impact of specific reforms and compare the effectiveness of each. Preliminary measures of efficiency have been proposed for patients with specific diseases, but, to our knowledge, none have been proposed to reflect a physician’s practice as a whole.3 The objective of this study is to develop an index to estimate the efficiency of ophthalmology practices with variables representative of practice setting, quality of care, number of patients receiving care, and resources used.
A metric of efficiency will allow the health care system to better measure the impact of specific reforms and compare the effectiveness of each.
An efficiency index is proposed, with variables representative of practice setting, quality of care, numbers of patients receiving care, and resources used.
The efficiency index is a metric that gives a broad overview of performance for a variety of ophthalmology specialties.
The following equation was used to calculate the efficiency index (E):
The efficiency index was defined as a function of adjusted costs (Ca), adjusted number of patients (Na), and quality (Q). Constant b is an empirically determined adjustment factor so that the value of E remains between the limits of 0 and 1. Constant y was set at a value of 2. Higher values of y will further differentiate practices with low quality from those with higher quality. For example, if 2 practices had Q1 = 0.1 and Q2 = 1, y = 0.5 would result in effective Q values of 0.3 and 1, whereas y = 2 would result in effective Q values of 0.01 and 1. In effect, the higher the value of y, the more the practice with lower quality will be punished. The units of b are such that E is a value without units.
The adjusted number of patients (Na) was calculated from the numbers of follow-up patients (N1), new patients (N2), and surgical patients (N3). Adjustments for each type of patient (x1, x2, x3) were empirically derived from Medicare relative value units (RVUs) for each ophthalmology subspecialty, which reflect the type of examinations and surgical procedures typically performed on patients:
Subspecialties in this study included cornea, comprehensive, glaucoma, neuro-ophthalmology, oculoplastics, pediatrics, surgical retina, medical retina, and uveitis.
Adjusted cost (Ca) is a variable sensitive to the location of the practice and describes the ratio of the cost of caring for a given group of patients to the value of the care provided as determined by Medicare RVUs:
Medicare value of work is a function of the number of patients, RVUs for each category of patients previously mentioned, and geographic practice cost indices. The RVUs and geographic practice cost indices are determined by the Centers for Medicare and Medicaid Services (CMS) for each fiscal year. A more detailed explanation of the calculation for Medicare value of work is available in the eEquation in the Supplement. In total, 7 inputs from the physician’s practice were required for the calculation of adjusted costs: (1) subspecialty; (2) location; (3) total number of new patients for a given period; (4) total number of follow-up patients for the same period; (5) total number of surgical patients for the same period; (6) total practice cost for that period (to care for the reported group of patients); and (7) the period under examination.
Quality (Q) in this study was a metric of medical process, as defined by the Donabedian model, and was scored by independent auditors masked to patient identities.6 For each practice, auditors used 20 randomly selected patient medical records and used patient-oriented quality questionnaires based on American Academy of Ophthalmology Preferred Practice Pattern.7 Ten medical records were used to complete a questionnaire pertaining to a comprehensive eye examination evaluation, and the other 10 medical records were used to complete a questionnaire from 1 of 3 diagnostic checklists as chosen by the practice depending on its subspecialty or stated area of interest or expertise. The 3 diagnostic checklists were specific to patients with cataract, patients with glaucoma, or patients with age-related macular degeneration.
A separate office-process questionnaire was developed by the American Academy of Ophthalmology Committee for Practice Improvement (J.C., chair) to address office processes beyond the scope of ophthalmic disease and did not require patient medical records. This questionnaire mainly focuses on front office performance, workflow, patient communication, incorporation of technology, and safety. For example, this questionnaire evaluates whether patients were seen in a timely manner and whether patients were greeted politely. It also takes into consideration whether an appropriate time-out was performed prior to surgical procedures. Scores from the 3 questionnaires were scored as a percentage, averaged, and then converted to a single value score of Q between 0 and 1 for each practice. The questionnaires are included in the eAppendix in the Supplement.
For the purposes of this article, a practice is defined as that of a single physician. For groups of physicians, cost and revenue were separated by those incurred by each physician to calculate the efficiency index. The period of interest for this pilot study was the 2012 fiscal year (October 2011 to September 2012). This study was approved by the University of California, Los Angeles Human Research Program and all methods adhered to Health Insurance Portability and Accountability Act of 1996 regulations and the tenets of the Declaration of Helsinki.
For the pilot study, 36 practices across 9 specialties in southern California were included. For these practices, constant b was empirically determined to be 7.0 × 10−5 so that the efficiency index would have a range from 0 to 1. For this pilot study, y was arbitrarily set at a value of 2. eTable 1 and eTable 2 in the Supplement show the values used to calculate the adjusted number of patients as well as CMS values for the 2012 fiscal year. The median adjusted number of patients was 5516 (interquartile range, 3450-11 863) (Figure 1). The median adjusted cost was 1.34 (interquartile range, 0.99-1.96), and Figure 2 shows a breakdown of this variable by comparing the total value of work determined by CMS RVUs to actual practice costs. The median quality score was 0.89 (interquartile range, 0.79-0.91) (Figure 3). The median efficiency was 0.26 (interquartile range, 0.08-0.42) (Figure 4). For 5 of the practices, 3 independent raters measured quality with 10 different medical records selected randomly (average coefficient of variation, 13.3%). The Table shows the summary of the efficiency index scores and parameters. eTable 3 in the Supplement provides examples for the values of each variable used to calculate the efficiency index.
In this pilot study, we propose an efficiency index estimated by the cost to run a practice for a measured number of patients as well as the quality of care (process only) provided to patients and apply the index to a broad range of ophthalmology subspecialties in an academic eye institute in southern California. The efficiency index is a metric that gives a broad overview of performance for a variety of ophthalmology specialties as measured by resources used and a preliminary measure for quality of care provided. Although all the practices in this pilot study were from the same region, the efficiency index also contains variables calculated by CMS sensitive to the geographic location of the practice. The geographic practice cost index for each RVU component was established for 89 different geographic locations by CMS for 2012.8 To our knowledge, no previous study has integrated location, cost, quality of care, and physician performance to evaluate physician practices. As health care reforms are implemented, a metric that incorporates cost will be required to support efforts to increase the value of health care. It will no longer be enough to merely deliver the best care; a more important goal will be to deliver quality care at the lowest cost.9
The efficiency index proposed in this study accounts for differences in patient populations based on the kind of examinations and types of surgical procedures typically performed for each subspecialty. By weighing different types of patients differently, the efficiency index accounts for some of the different complexities of each type (new, follow-up, and surgical patients) and the changes in the ratio of each type over time. Currently, the adjustments assume a representative mix of patients for each subspecialty; however, these values can be further refined with the analysis of additional practices. The index also takes into account the geographic location of the practice. By normalizing actual practice costs to CMS-determined values based on geographic location, the efficiency index mitigates the effects on costs of different patient populations and allows for a potential comparison between practices. The boundaries of these areas are determined by CMS and reflect the differences in health care for different regions of the United States. All practices were located in southern California in this pilot study, but the values for the geographic-specific variables are available for the entire country.
The efficiency index was developed to compare scores prior to and after training modules or changes in practice policies to gauge the effectiveness of an intervention. It is important to note that this efficiency index was designed for intrapractice comparisons before and after interventions. However, such an index obviously has the potential to compare performance between different subspecialties, after more work is done to give balanced weight to subspecialty-specific variables (x1, x2, x3) and to patient mix regarding severity and complexity of disease, so that performances of different practices may be compared fairly. When performance measures can be used for interpractice observations, physicians may learn from others with high efficiency how they structure their clinical practices and optimize their own to become more efficient. Further investigation will have to be performed to determine the sensitivity of the efficiency index to changes made to a practice.
In the Donabedian model, quality can be represented by 3 components: structure, process, and outcome.6 Structure refers to the characteristic of the place in which care is provided and consists of facilities, human resources, etc. Process refers to the actions taken in providing care to the patient. Outcome refers to the health status and, to a degree, patient experience. According to the Donabedian model, all 3 components are linked: “good structure increases the likelihood of good process, and good process increases the likelihood of a good outcome,”6 with process and outcome being the most important components.
Various approaches to estimate the different components of quality have been used. Prior studies have used survey results from peers or patients to measure physician performance, but these metrics do not reflect evidence-based medicine.10- 12 Physician performance has also been graded on outcomes. For example, CMS offers incentives to physicians who report their outcomes based on the Physician Quality Reporting System, which consists of more than 200 measures spread across various specialties. However, outcome measures do not exist for every condition and may shift focus to patients with measurable outcomes and dampen motivation to perform well for those without.13 Outcome measures, if not applied correctly, may place more focus on measurable tasks and detract physicians from the creative and complex problem solving that is essential in patient care.14 Furthermore, outcome-based measures may not completely capture variations in patients with the same disease. If the overall disease severity of one practice’s patient population differs from another, outcome measures would inaccurately reflect a physician’s skills and abilities.12 Risk adjustment is another method to make practices with different patient populations more comparable, but too much risk adjustment may eliminate important differences. Further work is needed to balance the amount of risk adjustment required to evaluate different outcomes so that physicians are not punished for having sicker patients and differences in physician performance can be meaningfully measured.
The preliminary measure of quality in this efficiency index is based on process alone. It reflects a component that lays a foundation for good outcomes, but it does not measure outcomes directly. As such, it does not reflect a physician’s performance in its entirety, but it does not have the previously mentioned pitfalls of outcome-based measures. Because practice guidelines for ophthalmology have been long established, the efficiency index can assess an ophthalmologist’s performance for a practice as a whole and should not undermine the care of certain patients. However, evidence-based guidelines do not exist for every disease, and for diseases without well-established guidelines, only assessing patients with conditions that have evidence-based measures may not accurately reflect the scope of a physician’s practice.12 Because practice guidelines do not always translate into better patient outcomes, future work must focus on incorporating appropriately adjusted outcome measures, including patient experience, to more fully evaluate quality.15 The American Academy of Ophthalmology Intelligent Resource in Sight registry is a database that may in the future help us better understand which clinical outcomes are important for specific ophthalmic conditions.
In the efficiency index, the value of Q was raised to the power of y. Increasing values of y will penalize practices with lower quality scores. For example, if 2 practices had quality scores of 1 and 0.1, a value of 1 for y would result in quality scores of 1 and 0.1, whereas a value of 2 for y would result in scores of 1 and 0.01—which relatively rewards by 10-fold the practice with the higher Q. As mentioned earlier, for this pilot study, y was arbitrarily set at a value of 2. The amount at which quality of health care should be weighted against cost is a societal decision, but it is our opinion that efforts to reduce cost should not come at the expense of quality.
As health care in the United States increasingly emphasizes cost, physicians will need a method to objectively assess their performance to move to a more value-based system. In this pilot study, we propose an efficiency index that estimates the efficiency of an ophthalmology practice with variables representing practice setting, quality of process of care, and resources used. While this index is designed to compare the efficiency of a practice with itself after an intervention, future indices would serve the dual purpose of providing data for quality improvement as well as for moving the health care model to a more efficient system.
Corresponding Author: Joseph Caprioli, MD, Department of Ophthalmology, David Geffen School of Medicine, University of California, Los Angeles, 100 Stein Plaza, Los Angeles, CA 90095 (firstname.lastname@example.org).
Submitted for Publication: December 20, 2014; final revision received March 20, 2015; accepted March 23, 2015.
Published Online: May 28, 2015. doi:10.1001/jamaophthalmol.2015.1447.
Author Contributions: Dr Caprioli had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Afifi, Caprioli.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Chen, Afifi, Caprioli.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Chen, Aigner, Afifi, Caprioli.
Obtained funding: Caprioli.
Administrative, technical, or material support: Chen, Kim.
Study supervision: Caprioli.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
Funding/Support: This work was supported by a grant from the American Academy of Ophthalmology.
Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: We thank the members of the American Academy of Ophthalmology Committee for Practice Improvement (Anthony C. Arnold, MD, Department of Ophthalmology, David Geffen School of Medicine, University of California Los Angeles; Steven V. L. Brown, MD, Chicago Glaucoma Consultants, Chicago, Illinois; David A. Durfee, MD, American Academy of Ophthalmology, San Francisco, California; Bradley D. Fouraker, MD, Brandon Cataract Center and Eye Clinics, Brandon, Florida; James C. Orcutt, MD, PhD, Department of Ophthalmology, University of Washington, Seattle; and Beth Wilson, American Academy of Ophthalmology, San Francisco, California) for their contributions to the development of the efficiency index; no compensation was received for these contributions.