0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Variation in Quality of Urgent Health Care Provided During Commercial Virtual Visits

Adam J. Schoenfeld, MD1,2; Jason M. Davies, MD, PhD1,2,3; Ben J. Marafino, BS1,2; Mitzi Dean, MS, MHA1,2; Colette DeJong, BA1,2,4; Naomi S. Bardach, MD, MAS1,2,5; Dhruv S. Kazi, MD, MS2,6; W. John Boscardin, PhD6,7; Grace A. Lin, MD, MAS1,2,6; Reena Duseja, MD2,7; Y. John Mei, AB1,2,6; Ateev Mehrotra, MD, MPH8; R. Adams Dudley, MD, MBA1,2,6,9
[+] Author Affiliations
1Center for Healthcare Value, University of California, San Francisco (UCSF)
2Philip R. Lee Institute for Health Policy Studies, UCSF
3Department of Neurosurgery, UCSF
4Department of Medicine, UC (University of California) Berkeley–UCSF Joint Medical Program
5Department of Pediatrics, UCSF
6Department of Medicine, UCSF
7Department of Emergency Medicine, UCSF
8Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
9Department of Epidemiology and Biostatistics, UCSF
JAMA Intern Med. 2016;176(5):635-642. doi:10.1001/jamainternmed.2015.8248.
Text Size: A A A
Published online

Importance  Commercial virtual visits are an increasingly popular model of health care for the management of common acute illnesses. In commercial virtual visits, patients access a website to be connected synchronously—via videoconference, telephone, or webchat—to a physician with whom they have no prior relationship. To date, whether the care delivered through those websites is similar or quality varies among the sites has not been assessed.

Objective  To assess the variation in the quality of urgent health care among virtual visit companies.

Design, Setting, and Participants  This audit study used 67 trained standardized patients who presented to commercial virtual visit companies with the following 6 common acute illnesses: ankle pain, streptococcal pharyngitis, viral pharyngitis, acute rhinosinusitis, low back pain, and recurrent female urinary tract infection. The 8 commercial virtual visit websites with the highest web traffic were selected for audit, for a total of 599 visits. Data were collected from May 1, 2013, to July 30, 2014, and analyzed from July 1, 2014, to September 1, 2015.

Main Outcomes and Measures  Completeness of histories and physical examinations, the correct diagnosis (vs an incorrect or no diagnosis), and adherence to guidelines of key management decisions.

Results  Sixty-seven standardized patients completed 599 commercial virtual visits during the study period. Histories and physical examinations were complete in 417 visits (69.6%; 95% CI, 67.7%-71.6%); diagnoses were correctly named in 458 visits (76.5%; 95% CI, 72.9%-79.9%), and key management decisions were adherent to guidelines in 325 visits (54.3%; 95% CI, 50.2%-58.3%). Rates of guideline-adherent care ranged from 206 visits (34.4%) to 396 visits (66.1%) across the 8 websites. Variation across websites was significantly greater for viral pharyngitis and acute rhinosinusitis (adjusted rates, 12.8% to 82.1%) than for streptococcal pharyngitis and low back pain (adjusted rates, 74.6% to 96.5%) or ankle pain and recurrent urinary tract infection (adjusted rates, 3.4% to 40.4%). No statistically significant variation in guideline adherence by mode of communication (videoconference vs telephone vs webchat) was found.

Conclusions and Relevance  Significant variation in quality was found among companies providing virtual visits for management of common acute illnesses. More variation was found in performance for some conditions than for others, but no variation by mode of communication.

Figures in this Article

Figures

Place holder to copy figure label and caption
Figure 1.
Completeness of History and Physical Examination by Condition and Virtual Visit Company

Each data point represents the adjusted mean rate of completeness by condition across all virtual visit companies (A) and by virtual visit company across all conditions (B). The error bars indicate 95% CIs; dotted line, the aggregate mean across conditions or virtual visit companies. Variations in completeness by condition and by virtual visit company were statistically significant (P < .001). UTI indicates urinary tract infection.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
Rate of Physician Naming the Correct Diagnosis by Condition and by Virtual Visit Company

Rates of naming the correct diagnosis for each visit are based on whether the physician stated the correct diagnosis for each encounter. Each data point represents the adjusted mean rate of naming the correct diagnosis by condition across all virtual visit companies (A) and by virtual visit company across all conditions (B). The error bars indicate the 95% CIs; dotted line, the aggregate mean across conditions or virtual visit companies. Variations in naming the correct diagnosis by condition and by virtual visit company were statistically significant (P < .001). UTI indicates urinary tract infection.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 3.
Adherence to Guidelines for Key Management Decisions by Condition and by Virtual Visit Company

Each point represents the adjusted mean rate of adherence by condition across all virtual visit companies (A) and by virtual visit company across all conditions (B). The error bars indicate 95% CIs; dotted line, the aggregate mean across conditions or virtual visit companies. Variation in guideline adherence was statistically significant by condition (P < .001) and virtual visit company (P = .009). UTI indicates urinary tract infection.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 4.
Variation by Pairs of Conditions Among Virtual Visit Companies in Adherence to Guidelines for Key Management Decisions

Each point represents the adjusted mean rate of adherence to guidelines in key management decisions for streptococcal pharyngitis and low back pain (best adherence [A]), for ankle pain and recurrent female urinary tract infection (UTI) (lowest adherence [B]), and for viral pharyngitis and acute rhinosinusitis (intermediate adherence [C]) for each virtual visit company. The error bars indicate 95% CIs; dotted line, the aggregate mean across virtual visit companies. Lower rates indicate lower adherence to guidelines in management decisions. Variation between virtual visit companies in adherence to guidelines was not statistically significant for streptococcal pharyngitis and low back pain (P = .29) or for ankle pain and UTI (P = .33); variation was significant for viral pharyngitis and acute rhinosinusitis (P < .001).

Graphic Jump Location

Tables

References

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment
Lacks Comparison to In-Person Visits
Posted on April 29, 2016
Jonathan D. Linkous
American Telemedicine Association
Conflict of Interest: Chief Executive Officer, American Telemedicine Association
The recent study, Variation in Quality of Urgent Health Care Provided During Commercial Visits , provides an important first look at the growing use of the internet to deliver consultations for non-emergency acute illnesses. The American Telemedicine Association (ATA) estimates that there will be 1,250,000 such encounters this year alone.
The findings of variation are disturbing and deserve further investigation. However, there is nothing in the article to suggest that variations in quality or adherence to standards within telehealth are any different than the variations seen within traditional in-person medicine. It is now well understood that variations in care exist across all of medicine (when comparing hospitals, medical offices, geographies, doctors within a single office setting, etc.). The fact that this was observed among telehealth organizations should not be surprising at all. Statistically significant variation between providers of virtual visits is only significant if that variation differs from in-person care.
Regardless of the variation, as virtual visits become an important component in the delivery of care implementation of guidelines and standards are paramount. In response, last year ATA launched an accreditation program for sites providing virtual consultations to help assure patient safety, transparency of operations and adherence to relevant laws and regulations. One essential step toward accreditation is assuring network providers adhere to established guidelines in decision making. Almost 300 sites have been determined to be eligible for accreditation.
We hope studies like this bring on more research and subsequent opportunity to bring improvement to telemedicine. We need more data, and the funding to study it, to fuel the growth and adoption of telehealth.
Alternative method of virtual care provides superior adherence to guidelines
Posted on May 11, 2016
Rebecca Hafner-Fogarty, MD, MBA, FAAFP,Kevin L. Smith, DNP, FNP, FAANP
Zipnosis, Inc.
Conflict of Interest: Both authors are employees of Zipnosis, Inc.
The original investigation article “Variation in Quality of Urgent Health Care Provided During Virtual Visits” adds much-needed data to the limited body of research available in the rapidly evolving field of virtual health care.

While the virtual care company that we represent, Zipnosis, was not included in the study, we read with interest the wide variation in quality that Schoenfeld and colleagues identified among 8 virtual visit services treating 6 common acute illnesses. Of note, the companies in the study primarily provide outsourced care from providers outside of the patient’s health system — a fragmented model that may have affected quality and guideline adherence. They also rely on synchronous (live) systems, such as direct-to-video-based telemedicine.

An alternative to this model that provides continuity of care is an asynchronous virtual health platform connecting patients directly to local providers within their community’s health system — typically MDs/DOs/NPs/PAs staffing its urgent care centers with access to the patient’s EMR. This innovative approach to mainstream medicine leverages existing clinical capacity and supports efficient, high-quality care. In addition, unlike live video-based telemedicine, an asynchronous (store-and-forward) system means the patient’s information is added to the clinical notes prior to assessment — eliminating the possibility of a busy clinician missing a key question.

This model guides patients through structured, adaptive interviews developed from national best practice guidelines. A systematic method also leads providers through curated pathways based on patients’ medical histories. Treatment options are limited to the most appropriate choices supported by evidence-based protocols, thereby reducing variability.

This approach results in consistently high-quality care, especially when compared with the outcomes reported in the Schoenfeld study, which revealed guideline-adherent care ranging from 34.4% to 66.1%. Variation among the 8 companies included in the study was even greater for viral pharyngitis and acute rhinosinusitis, at 12.8% to 82.1%.

In contrast, data collected from 1,760 patient visits at two large U.S. health systems using the Zipnosis platform demonstrated a >95% adherence to national best practice guidelines(1,2) for the treatment of acute rhinosinusitis. These data have been reported to a state medical board and presented at major informatics and telehealth conferences.(3,4)

With each virtual visit saving on average $175, the delivery of convenient, guideline-adherent care by local providers embodies the Institute for Healthcare Improvement’s “Triple Aim”: To optimize health system performance by improving patient experience, improving population health, and reducing the per capita cost of healthcare.

REFERENCES
1. Rosenfeld RM, Piccirillo JF, Chandrasekhar SS, et al. Clinical practice guideline (update): adult sinusitis. Otolaryngol Head Neck Surg. 2015;152(2 Suppl):S1–S39.
2. Chow AW, Benninger MS, Brook I, et al; for the Infectious Diseases Society of America (IDSA). IDSA clinical practice guideline for acute bacterial rhinosinusitis in children and adults. Clin Infect Dis. 2012;54(8):1041–1045.
3. Smith KL. Adherence to sinusitis treatment guidelines in the evisit setting: a quality improvement project. Presented at: iHealth 2016 Clinical Informatics Conference [American Medical Informatics Association]; May 4–6, 2016. Minneapolis, MN.
4. Smith KL. Sinusitis treatment guideline adherence in the e-visit setting: a performance improvement project [Part of: Online care measurements and guidance session]. Presented at: American Telemedicine Association 2016 Annual Conference; May 14–17, 2016. Minneapolis, MN.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

2,686 Views
2 Citations
×

Sign in

Purchase Options

• Buy this article
• Subscribe to the journal
• Rent this article ?

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
Jobs
JAMAevidence.com

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Low Back Pain

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Low Back Pain, Disabling

brightcove.createExperiences();