(480) 923-0802

Our Take: Embracing the use of ChatGPT with enthusiasm and caution, preferably in equal measure

Jun 26, 2023

With the potential to dramatically transform countless aspects of health care, from communications and workflow to research, education, and diagnoses, ChatGPT — the artificial intelligence chatbot that Microsoft-backed OpenAI launched in November — has been the hot topic in health system C-suites for months.

While ChatGPT and similar large language models, such as Google Bard (LaMDA), hold seemingly boundless promise in health care at this point, there are also significant opportunities for calamitous outcomes. Consequently, opinions on the best approach for moving forward run the gamut.

Some have urged instituting a moratorium until the risks can be more fully assessed. Others are ready to fully explore all possibilities. Many are taking a stance somewhere in between, wanting to put guardrails in place before going all out and, for the time being, limiting use of the AI tools to administrative functions, such as scheduling appointments, requesting prior authorizations, automating billing and coding processes, and following up on claims.

In an effort to develop guardrails and guidelines for the responsible use of AI in health care, and to educate end-users so they can evaluate and promote the adoption of technologies such as ChatGPT, several academic health systems have come together with federal agencies and leading technology companies including Change Healthcare, Google, Microsoft, MITRE, and SAS Institute to create the Coalition for Health AI

After privacy, security, ethical and other concerns have been adequately addressed, clinical use of ChatGPT could go a long way toward alleviating clinician burnout. One day in the not too distant future, ChatGPT could — with human oversight — assume tasks such as maintaining clinical documentation, summarizing patients’ medical histories, and answering patients’ inquiries about symptoms, medications, and follow-up care.

In time — and some believe it will be a matter of months, not years — physicians could use ChatGPT to instantaneously sift through massive amounts of medical literature and access all of a patient’s medical history, imaging documents, genomics, and lab results with a single inquiry, giving them quicker access to data they need to make a diagnosis or choose a course of treatment.

But first, the predictive bot’s tendency to generate “hallucinations” when insufficient data is available must be eliminated. Hallucinations occur when ChatGPT pulls pieces of data from various sources and combines them into a response that seems legitimate but is factually incorrect. Not only does this inclination inherently risk patient safety, but bad actors can also exploit the weakness to wreak havoc in myriad ways.

As Dr. Neil Mehta, associate dean of curricular affairs at Cleveland Clinic Lerner College of Medicine of Case Western Reserve University said in a recent podcast, now that OpenAI has made ChatGPT publicly accessible free of charge (an updated model, GPT-4, requires a subscription), “the cat’s already out of the bag.” He pointed out, basically, that attempts to restrict the use of ChatGPT won’t deter those with less honorable intentions.

Epic may be the gateway for many hospitals and health systems to begin using ChatGPT. The company announced in April that it was expanding its collaboration with Microsoft by integrating generative AI solutions, like GPT-4, into its EHR platform.

Here is a sampling of how some hospitals and health systems are approaching the use of generative AI and chatbot technology such as ChatGPT:

  • Mayo Clinic is among several health care organizations testing Google Cloud’s Enterprise Search on Generative AI App Builder, a tool that lets users create custom, HIPAA-compliant chatbots and semantic search apps to efficiently review huge amounts of internal data.
  • Cleveland Clinic assessed ChatGPT’s ability to answer simple questions about preventing cardiovascular disease. Dr. Ashish Sarraju, who led the study, said the AI model performed better than expected, but Dr. Leslie Cho, a co-author of the study, said much more work is needed before the technology can be considered for use in clinical practice.
  • Kaiser Permanente has established the Augmented Intelligence in Medicine and Healthcare Initiative (AIM-HI). With support from the Gordon and Betty Moore Foundation, AIM-HI will award three to five health systems grants of up to $750,000 each for proposals on using AI and machine learning to enhance diagnostic decision making.
  • Although Advocate Health is exploring the use of AI to improve patient outcomes at its Advocate Aurora Research Institute (e.g., through KelaHealth’s Surgical Intelligence Platform), the health system has cautioned its clinicians against relying on ChatGPT to answer clinical questions when evidence-based information is required, advising them to continue using “proven medical information resources” provided by the health system’s medical library.
  • Vanderbilt Medical Center is testing ChatGPT to see how it can improve automated alerts that support clinical decision making, Becker’s Hospital Review reported.
  • UNC Health, UC San Diego Health, UW Health, and Stanford Health Care are participating in a pilot program using Epic and Microsoft’s generative AI to draft responses to patient inquiries in online portals.
  • Boston Children’s Hospital recently hired a ChatGPT expert who will identify use cases and train the hospital’s workforce on appropriate use of the technology.
  • Phoenix Children’s Hospital’s chief information officer, David Higginson, told MedCity News the hospital has warned its staff not to input any health records or personal information into ChatGPT but has not blocked the use of the technology. He said non-clinical staff is using large language models to make certain processes — like drafting letters to insurers — more efficient and to write educational materials for patient’s families.
  • The University of Miami Health System is using seminars, webinars, and workshops, along with written guidelines, to educate its staff on how to use the AI models responsibly.

Our Take: Some have speculated that the hype surrounding the use of ChatGPT in health care will soon die down. From what we’ve seen, that seems unlikely. There’s simply too much potential.

Health Care Rounds Episode 160: Closing Care Gaps With Data, with Ruth Krystopolski

One of the challenges of delivering health care in an equitable way lies in successfully identifying gaps in care and other disparities. Ayin Health Solutions, led by President Ruth Krystopolski, is closing those gaps by collecting, organizing, and bringing actionable data and other technology solutions to key industry stakeholders. Through this work, providers, payers, and delivery organizations can work together to reduce health disparities and improve the health of vulnerable populations. Renton, Wa.-based Providence launched Ayin Health Solutions in 2019.

What else you need to know
Ochsner Health and The University of Texas MD Anderson Cancer Center are partnering to create the Ochsner MD Anderson Cancer Center in southeastern Louisiana, which will offer a fully integrated cancer program based on MD Anderson’s standards and treatment plans. Seven of Ochsner’s cancer centers and medical centers in the New Orleans area, Baton Rouge, and Covington have been established as the first phase of co-branded Ochsner MD Anderson facilities, with the certification of more sites to follow. Along with comprehensive care for patients, the Ochsner MD Anderson program will provide access to “select clinical trials available to eligible patients as the only Phase I clinical trial program in Louisiana,” the organizations said in their announcement. The partnership marks UT MD Anderson’s seventh such collaboration with a major health system.

The FDA has approved the first gene therapy for Duchenne muscular dystrophy (DMD): Sarepta Therapeutics’ Elevidys (delandistrogene moxeparvovec-rokl). Currently, use of the therapy is restricted to patients ages 4 through 5 years who can still walk, have a confirmed mutation in the DMD gene, and have no genetic contraindications. Because Elevidys was approved via the FDA’s accelerated approval pathway, confirmatory clinical testing is required. Sarepta said in a press release it expects top-line results from an ongoing Phase III trial by year-end. If the therapy’s benefit is confirmed, the company can file for full approval with an expanded treatment population that includes older children. Although Elevidys is not a cure, it may improve physical function and mobility. The therapy is administered as a single infusion, which Sarepta has priced at $3.2 million.

Premier is divesting its non-health care group purchasing organization (GPO) operations for approximately $800 million in cash. The Charlotte, N.C.-based company signed a definitive agreement with Omnia Partners, which is also a non-health care GPO. Subject to regulatory approval and customary closing conditions, the transaction is expected to close by early August. If the deal is completed, Omnia will become the primary GPO for Premier’s non-health care members, which include educational, hospitality, and recreation organizations. Premier noted in the announcement that divesting the GPO operations will allow the company to focus on its member services and core health care businesses.

UPMC signed a non-binding letter of intent to integrate Washington Health System (WHS), a two-hospital, non-profit community health system based Washington, Pa., into the larger Pittsburgh-based integrated health system. The two organizations already have several clinical arrangements and joint ventures, as noted in a news release. In addition, WHS’ flagship hospital was an original member of the UPMC Health Plan when the health plan was launched 26 years ago. In the months ahead, the organizations will conduct due diligence and work toward a definitive agreement.

Eli Lilly signed a definitive agreement to acquire Dice Therapeutics for about $2.4 billion. Dice is a biopharmaceutical firm based in San Francisco that develops oral therapeutics for autoimmune diseases. Its lead candidates are oral antagonists of IL-17, and its proprietary Delscape platform is designed to discover selective oral small molecules with the potential to modulate protein-protein interactions as effectively as systemic biologics, according to a press statement. The acquisition has been approved by both companies’ boards and is expected to close in the third quarter if customary closing conditions are met. The agreed-upon purchase price of $48 per share represents a premium of approximately 40% to the 30-day volume-weighted average trading price of Dice’s common stock on the last day of trading before the deal was announced.

Seattle-based DexCare, a spinoff of Providence, closed a $75 million Series C funding round, bringing the total amount raised so far to $146 million. Kaiser Permanente Ventures was among several existing partners that participated in the latest funding round. DexCare’s platform, originally developed for Providence’s ambulatory care business, according to Healthcare Innovation, helps health systems manage the supply and demand of digital care access. DexCare CEO Derek Streat said in a press release, “With the backing of our investors, partners, and customers, we’re overhauling an antiquated industry to ensure patients are matched to the right provider.”

Boston Medical Center Health System’s board of trustees appointed Dr. Alastair Bell as the health system’s new CEO. Dr. Bell took on the role of president at BMC Health System in November and has been serving as interim CEO since March. He joined BMC Health System in 2012 as its chief operating officer.

What we’re reading 
2023 Scorecard on State Health System Performance. The Commonwealth Fund, 6.22.23
Value-Based Care Fuels Innovation, Not Consolidation. Health Affairs Forefront, 6.16.23
Unleashing CRISPR on Cancer. NEJM, 6.22.23


Contact Darwin Research Group and we will get right back to you.