CMS program participation and policy evaluation without administrative data: a case study on Bundled Payments for Care Improvement (BPCI) initiative
Original Article

CMS program participation and policy evaluation without administrative data: a case study on Bundled Payments for Care Improvement (BPCI) initiative

Ulysses Isidro1,2, Joseph R. Martinez1,2, Amol S. Navathe1,2

1Center for Health Incentives and Behavioral Economics at the Leonard Davis Institute of Health Economics at the University of Pennsylvania, Philadelphia, PA, USA; 2Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA

Contributions: (I) Conception and design: All authors; (II) Administrative support: AS Navathe; (III) Provision of study materials or patients: AS Navathe; (IV) Collection and assembly of data: U Isidro, JR Martinez; (V) Data analysis and interpretation: All authors; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

Correspondence to: Ulysses Isidro. University of Pennsylvania, 1118 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104-6021, USA. Email: ulysses.isidro@pennmedicine.upenn.edu.

Background: The US spends significantly more per capita on healthcare than other developed countries. The Centers for Medicare and Medicaid Services (CMS) Innovation Center (CMMI) has created various alternative payment models (APMs) that use financial incentives to reward providers for delivering higher value care, including bundled payments. In 2013 and 2018, CMS scaled up its bundled payment APM nationwide through its Bundled Payments for Care Improvement (BPCI) Initiative and BPCI Advanced Initiative, respectively. Studies of the effects of physician group practice (PGPs) participation have been delayed in part due to a lack lists of participating physicians available via CMS.

Methods: To assess whether health policy researchers could adequately evaluate the impact of BPCI without CMS administrative data, we investigated the accuracy of using non-CMS sources to identify BPCI physicians. Our researcher-created database (“Other Data Source List” or “ODSL”) of individual physicians participating in BPCI through a PGP was compared to a novel data set—a list of physicians in PGPs participating in BPCI directly from CMS (“CMS List”). We performed chi-squared tests to determine whether ODSL-identified physicians differed meaningfully from CMS List-identified physicians.

Results: Sixty-two percent of ODSL physicians were found in the CMS List of participating BPCI physicians, and ODSL contained 46% of BPCI physicians identified in the CMS List. ODSL was statistically different from the CMS List and had significant limitations in identifying participating BPCI physicians.

Conclusions: Policy evaluations that rely on identifying physicians using non-CMS sources may have a large degree of inaccuracy. If these challenges extend to other APMs, policy evaluations of such programs using non-CMS sources may also be inaccurate.

Keywords: Bundled payments; Bundled Payments for Care Improvement (BPCI); policy evaluation; physician group practices


Received: 20 May 2019; Accepted: 12 June 2019; Published: 15 July 2019.

doi: 10.21037/jhmhp.2019.06.03


Introduction

The US spends significantly more per capita on healthcare than other developed countries. Some reasons for this include lack of health insurance coverage, increased hospital fees due to recent hospital consolidation, high drug prices, and wasteful spending. The Centers for Medicare and Medicaid Services (CMS) has attempted to solve this problem by increasing the value of US healthcare through improvements in health outcomes and/or decreases in health-related costs. Specifically, CMS has created various alternative payment models (APMs) that use financial incentives to reward providers for delivering higher value care, including population-based shared savings programs, patient centered medical home models, and bundled payments.

Recently, CMS announced a greater emphasis on physician leadership in APMs (1). In 2013, CMS expanded its bundled payment APM through the Bundled Payments for Care Improvement (BPCI) Initiative for participating acute care hospitals and physician group practices (PGPs). BPCI builds on Medicare’s prospective payment system by paying a lump sum to providers for not only the acute care hospital stay, but also physician payments and all other spending up to 90 days after a hospitalization to encourage providers to coordinate their services and be more cost-conscious. Bundled payments in theory increase value by decreasing unnecessary healthcare costs, improving quality of care, and improving patient outcomes.

In 2018, CMS continued expanding its bundled payment APM through the BPCI Advanced Initiative (2). Although the impact of BPCI on participating hospitals has been evaluated, an understanding of the impact on PGPs has lagged, in large part due to an initial lack of available CMS administrative lists of participating physicians (3). To assess whether health policy researchers could adequately evaluate the impact of BPCI on PGP physicians without CMS administrative data, we investigated the accuracy of using data from available non-CMS sources to identify BPCI physicians.


Methods

National Provider Identifiers (NPIs) for individual physicians participating in phase 2 of BPCI model 2 through a PGP were first identified. To collect physician NPIs, we used non-CMS sources, specifically manual website searches of each PGP, the SK&A office-based physician dataset, and the NPI registry. We then assigned NPIs to PGPs based on organization name and address to create the “Other Data Source List” (“ODSL”). CMS subsequently made lists available that included physicians participating in BPCI for 2015 and 2016 (“CMS List”); comparing allowed evaluation of the accuracy of ODSL. We restricted both CMS List and ODSL to PGPs participating in the largest single episode (major joint replacement of the lower extremity) and then compared physician characteristics by linking information from the SK&A database. We performed chi-squared tests to determine whether ODSL-identified physicians were meaningfully different from CMS List-identified physicians.


Results

ODSL included 8,757 physicians, while the CMS List included 11,758 physicians. Sixty-two percent of ODSL physicians were found in the CMS List of participating BPCI physicians, and ODSL contained 46% of BPCI physicians identified in the CMS List (Table 1). Chi-squared tests performed by specialty, geography, and PGP size rejected equivalence of ODSL and CMS list (P<0.001) (Table 2). ODSL was statistically different from the CMS List and had significant limitations in identifying participating BPCI physicians.

Table 1
Table 1 PGP physician NPI LEJR-SKA match accuracy
Full table
Table 2
Table 2 ODSL vs. CMS List by physician characteristics
Full table

Specialty

On average, 71% of ODSL-identified physicians were missing from the CMS List, while 47% of CMS List-identified physicians were missing from ODSL. Of note, 43% of ODSL-identified internists, 20% of orthopedic surgeons, and 25% of physical medicine/rehab specialists were missing from the CMS List.

Geography

On average in each state and region (Northeast, Midwest, South, West), 43% (3–100%) and 44% (32–53%), respectively, of ODSL-identified physicians were missing from the CMS List, while 61% (20–100%) and 54% (49–60%), respectively, of CMS List-identified physicians were missing from ODSL.

PGP size

On average in large (50+ physicians), medium (10–49 physicians), and small PGPs (<10 physicians), 22%, 53%, and 74% (each 0–100%), respectively, of ODSL-identified physicians were missing from the CMS List, while 49% (10–88%), 27% (0–92%), and 14% (0–93%), respectively, of CMS List-identified physicians were missing from ODSL.


Discussion

In this study, we examined physician group participation in Medicare’s BPCI program and found that publicly available data could not be used to accurately identify a large proportion of participants. This suggests that the health policy research community is heavily reliant on the release of such data by regulatory agencies such as CMS to provide policy relevant analysis of program impact. Researchers frequently use primary data collection such as web scraping and other manual means to collect this information, as we did in this study, to fill in gaps in availability of data. While this is generally well-intentioned and noted as a limitation of research, this is the first study to our knowledge to compare a detailed effort to collate participation lists using publicly available data to participation lists made available by CMS. Unfortunately, our analysis did not corroborate that manual efforts are accurate.

This limitation of manual collection may not be limited to the BPCI program. There are several programs, unlike BPCI, for which the participation lists have not been made available to the research community. For example, to date, the Next Generation ACO program and several primary care initiatives do not provide lists with identifiers that can be used reliably by researchers (4-6).


Conclusions

Policy evaluations that rely on identifying physicians using non-CMS sources may have a large degree of inaccuracy. If these challenges extend to other CMS APMs (e.g., Next Generation Accountable Care Organizations), policy evaluations of such programs using non-CMS sources may also have large degrees of inaccuracy. Before CMS continues expanding its bundled payment APM in PGPs, robust research should be conducted to evaluate the effects of BPCI. However, if CMS does not publicly release and update lists of physicians participating in PGPs in a timely fashion, health policy researchers cannot accurately study the impact of BPCI on PGP physicians. Without such research, CMS will be unable to make an evidence-based decision to continue expanding its bundled payment programs. We expect that the effects of BPCI will be different on PGPs than on hospitals, similar to what has been observed in other APMs (7-9).


Acknowledgments

The Perelman School of Medicine at the University of Pennsylvania Health Services Research Summer Scholars Program sponsored by the Department of Medicine in collaboration with the Leonard Davis Institute of Health Economics and the Master of Science in Health Policy Research Program.


Footnote

Conflicts of Interest: Dr. Navathe reports receiving grants from Hawaii Medical Services Association, Anthem Public Policy Institute, Cigna, and Oscar Health; personal fees from Navvis and Company, Navigant Inc., Lynx Medical, Indegene Inc., Sutherland Global Services, and Agathos, Inc.; personal fees and equity from NavaHealth; personal fees from the National University Health System of Singapore; speaking fees from the Cleveland Clinic; serving as a board member of Integrated Services Inc., a subsidiary of Hawaii Medical Services Association, without compensation, and an honorarium from Elsevier Press, none of which are related to this manuscript. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.


References

  1. Verma S. “Medicare and Medicaid Need Innovation.” Wall Street Journal [published 19 September 2017, accessed 2 January 2018]. Available online: https://www.wsj.com/articles/medicare-and-medicaid-need-innovation-1505862017
  2. The Centers for Medicare & Medicaid Services. CMS Announces Participants in New Value-Based Bundled Payment Model. [Published 9 October 2018, accessed 22 October 2018] Available online: https://www.cms.gov/newsroom/press-releases/cms-announces-participants-new-value-based-bundled-payment-model
  3. The Lewin Group. CMS Bundled Payments for Care Improvement Initiative Models 2-4: Year 3 Evaluation & Monitoring Annual Report. [Published October 2017, accessed 22 October 2018] Available online: https://downloads.cms.gov/files/cmmi/bpci-models2-4yr3evalrpt.pdf
  4. The Centers for Medicare & Medicaid Services. Comprehensive Primary Care Initiative. [Updated 14 May 2019, accessed 07 June 2019] Available online: https://innovation.cms.gov/initiatives/comprehensive-primary-care-initiative/
  5. The Centers for Medicare & Medicaid Services. Comprehensive Primary Care Plus. [Updated 14 May 2019, accessed 07 June 2019] Available online: https://innovation.cms.gov/initiatives/comprehensive-primary-care-plus/
  6. The Centers for Medicare & Medicaid Services. Next Generation ACO Model. [Updated 21 May 2019, accessed 07 June 2019] Available online: https://innovation.cms.gov/initiatives/next-generation-aco-model/
  7. McWilliams JM, Hatfield LA, Landon BE, et al. Medicare Spending after 3 Years of the Medicare Shared Savings Program. N Engl J Med 2018;379:1139-49. [Crossref] [PubMed]
  8. Muhlestein D, Saunders RS, McClellan MB. Medicare Accountable Care Organization Results For 2015: The Journey To Better Quality And Lower Costs Continues [published 09 September 2016, accessed 22 October 2018. Available online: https://www.healthaffairs.org/do/10.1377/hblog20160909.056418/full/
  9. Saunders RS, Muhlestein D, & McClellan MB. Medicare Accountable Care Organization Results For 2016: Seeing Improvement, Transformation Takes Time. (published 21 November 2017, accessed 22 October 2018) Available online: https://www.healthaffairs.org/do/10.1377/hblog20171120.211043/full/
doi: 10.21037/jhmhp.2019.06.03
Cite this article as: Isidro U, Martinez JR, Navathe AS. CMS program participation and policy evaluation without administrative data: a case study on Bundled Payments for Care Improvement (BPCI) initiative. J Hosp Manag Health Policy 2019;3:16.