Telehealth measurement has recently received renewed attention as healthcare organizations rapidly adopted and deployed telehealth programs during the COVID-19 pandemic. Since the pandemic started, the need to measure utilization and its temporal variations accurately has increased as the rate and type of telehealth visits grew substantially and is now stabilizing. Measures that interact with utilization, like cost, have also not been effectively quantified to understand the financial impact of telehealth utilization variation during and since the pandemic and are only recently being rigorously assessed. This may be due to challenges accessing complete, reliable data, especially at the organizational level. As health services researchers who conduct evaluations in various areas of telehealth, these limitations restrict how health services researchers, who conduct evaluations in various areas of telehealth, define and measure telehealth among hospitals to inform accurate comparisons of utilization and care provision via telehealth. Thus, we spend much of our time asking one key question: how do we effectively and comprehensively measure telehealth? This question has led us to think more critically about the nuances of telehealth measurement. Specifically, should these measures be assessed at the organizational/hospital level or hierarchically with individual-level data in mind? Is analysis done best across hospitals that report longitudinal utilization data? More importantly, what services—delivered by whom—should be quantified given the substitutive effect of care from brick-and-mortar settings to direct-to-consumer applications?
Understanding How Disruptions to Care Modalities Disrupt Data Capture and Measurement
The swift expansion of telehealth services in response to COVID-19 was not just a necessary adaptation to traditional in-person care. A seismic shift illuminated telehealth’s capacity to reshape healthcare accessibility and efficiency in good and bad ways. Based on prior research by researchers at the Medical University of South Carolina (MUSC), this was most evident among rural hospitals where consistent, longitudinal comparisons of telehealth utilization across years were limited. Despite rapid adoption outpacing our ability to comprehensively measure telehealth utilization, we were motivated to investigate barriers and facilitators, beginning with available data sources. First, we sought to determine what organizational-level data are available to measure telehealth utilization across years and what the barriers (i.e., cost prohibitive, identifiable data, levels of missingness, reliability, and comprehensiveness) were to access such data. Common data sources used to longitudinally test such questions include the American Hospital Association (AHA) Annual Survey, the Healthcare Information and Management Systems Society (HIMSS) Dorenfest Institute Data, CMS Healthcare Cost Reporting Information System (HCRIS) cost reports, claims data, and certain industry-specific data. Using these sources, we've measured telehealth in different ways—all with limitations we believe can be improved upon. The purpose for which these diverse data are collected vary and how telehealth is defined is not standard across them. For example, telehealth indicators are populated by codes derived from CPT or HCPCS terminologies. Additional context was needed. We then contacted administrators and researchers reliant on these data sources and frameworks to guide telehealth measurement.
Evolving Telehealth Measurement Guidance
A common refrain in our discussions was that existing quality organizations provided the basis for utilizing available data to measure telehealth equitably. The National Quality Forum (NQF) developed a framework to guide telehealth quality measurement, focusing on four evaluation outcomes: care access, cost, experience, and effectiveness. While these variables have been examined at length in recent research, the interaction across these domains is underexplored. This is especially true of organizational financial dimensions of telehealth utilization. Many hospitals received COVID-19 funding from the Public Health Emergency (PHE) declaration and associated regulations, which had immediate and long-term effects on access, cost, experience, and effectiveness. The effects of this funding on organizational telemedicine service development and provision may inform future expansion and sustainability efforts. This use case underscores the importance of understanding the cross-cutting financial impact of one-time funding on hospitals. It may have served as a facilitating force for hospitals reporting telehealth utilization data to regulators and professional associations, which is essential for healthcare providers and policymakers to make informed decisions about integrating and expanding telehealth services.
Hospital Reporting and Research Gaps
These underexplored relationships represent a significant gap in what we know and do not know about the true magnitude of telemedicine’s ability to improve access, cost, quality, effectiveness, and experience. Professional associations that measure telehealth at hospitals through annual surveys like the AHA and HIMSS are limited in their ability to improve data capture because they cannot require hospitals to respond or validate accurate responses. Annual mandatory filings through CMS, such as cost reports, do not currently require organizations to report specific volumes or dollars of telehealth information. While claims data may be one approach to quantify telehealth provision in the future, the lack of consistent, required billing codes specific to telehealth prevents us from measuring telehealth historically. As a result, the latest research has focused on telehealth's efficacy and patient satisfaction, creating a conspicuous paucity of research that contextualizes organizational financial implications. This gap hinders our ability to fully appreciate the cost-benefit dynamics of telehealth and develop strategies that maximize its economic and clinical benefits.
Summary
As we transition into the post-pandemic era, we must forge ahead with more advanced, thoughtful methods for capturing and measuring telehealth utilization data and its financial impact by doing the following:
- Developing clear, reliable definitions of telemedicine
- Incentivizing reporting by hospitals and other healthcare delivery organizations
- Ensuring inclusive input from all stakeholders
- Providing an equitable reporting framework for organizational respondents of all types
These steps necessitate a collective effort from researchers, administrators, survey developers, healthcare providers, regulators, and policymakers to improve data collection practices. Only with complete and accurate data can we ensure that telehealth reaches its full potential as a pillar of modern healthcare.
This publication was made possible by the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services (HHS) as part of the National Telehealth Centers of Excellence Award (U66RH31458). The contents are those of the author(s) and do not necessarily represent the official views of nor an endorsement by the HRSA, HHS or the US Government.
