Ahead of a visit to the hospital for a surgical procedure, patients often have plenty of questions about what to expect — and can be plenty nervous.
To help minimize pre-surgery jitters, NVIDIA and Deloitte are developing AI agents using NVIDIA AI to bring the next generation of digital, frontline teammates to patients before they even step foot inside the hospital.
These virtual teammates can have natural, human-like conversations with patients, answer a wide range of questions and provide supporting guidance prior to preadmission appointments at hospitals.
This demo shows one virtual representative in action, answering patient questions:
Working with NVIDIA, Deloitte has developed Frontline AI Teammate for use in settings like hospitals, where the digital avatar can have practical conversations — in any language — that give the end user, such as a hospital patient, instant answers to pressing questions.
Powered by the NVIDIA AI Enterprise software platform, Frontline AI Teammate includes avatars, generative AI and large language models.
”Avatar-based conversational AI agents offer an incredible opportunity to reduce the productivity paradox that our healthcare system faces with digitization,” said Niraj Dalmia, partner, Deloitte Canada. “It could possibly be the complimentary innovation that reduces administrative burden, complements our healthcare human resources to free up capacity, and helps solve for patient experience challenges.”
Next-Gen Technologies Powering Digital Humans
Digital humans can provide lifelike interactions that can enhance experiences for doctors and patients.
Developers can tap into NVIDIA NIM microservices, which streamline the path for developing AI-powered applications and moving AI models into production, to craft digital humans for healthcare industry applications. NIM includes an easily adaptable NIM Agent Blueprint developers can use to create interactive, AI-driven avatars that are ideal for telehealth — as well as NVIDIA NeMo Retriever, an industry-leading embedding, retrieval and re-ranking model that allows for fast responses based on up-to-date healthcare data.
Customizable digital humans — like James, an interactive demo developed by NVIDIA — can handle tasks such as scheduling appointments, filling out intake forms and answering questions about upcoming health services. This can make healthcare services more efficient and more accessible to patients.
In addition to NIM microservices, James uses NVIDIA ACE and ElevenLabs digital human technologies to provide natural, low-latency responses.
NVIDIA ACE is a suite of AI, graphics and simulation technologies for bringing digital humans to life. It can integrate every aspect of a digital human into healthcare applications — from speech and translation abilities capable of understanding diverse accents and languages, to realistic animations of facial and body movements.
Deloitte’s Frontline AI Teammate, powered by the NVIDIA AI Enterprise platform and built on Deloitte’s Conversational AI Framework, is designed to deliver human-to-machine experiences in healthcare settings. Developed within the NVIDIA Omniverse platform, Deloitte’s lifelike avatar can respond to complex, domain-specific questions that are pivotal in healthcare delivery.
The avatar uses NVIDIA Riva for fluid, multilingual communication, helping ensure no patient is left behind due to language barriers. It’s also equipped with the NeMo Megatron-Turing 530B large language model for accurate understanding and processing of patient data. These advanced capabilities can make clinical visits less intimidating, especially for patients who may feel uneasy about medical environments.
Personalized Experiences for Hospital Patients
Patients can get overwhelmed with the amount of pre-operative information. Typically, they have only one preadmission appointment, many weeks before the surgery, which can leave them with lingering questions and escalating concerns. The stress of a serious diagnosis may prevent them from asking all the necessary questions during these brief interactions.
This can result in patients arriving unprepared for their preadmission appointments, lacking knowledge about the appointment’s purpose, duration, location and necessary documents, and potentially leading to delays or even rescheduling of their surgeries.
To enhance patient preparation and reduce pre-procedure anxiety, The Ottawa Hospital is using AI agents, powered by NVIDIA and Deloitte technologies, to provide more consistent, accurate and continuous access to information.
With the digital teammate, patients can experience benefits including:
- 24/7 access to the digital teammate using a smartphone, tablet or home computer.
- Reliable, preapproved answers to detailed questions, including information around anesthesia or the procedure itself.
- Post-surgery consultation to resolve any questions about the recovery process, potentially improving treatment adherence and health outcomes.
In user acceptance testing of the digital teammate conducted this summer, a majority of the testers noted that responses provided were clear, relevant and met the needs of the given interaction.
“The Frontline AI Teammate offers a novel and innovative solution to help combat our health human resource crisis — it has the potential to reduce the administrative burden, giving back time to healthcare providers to provide the quality care our population deserves and expects from The Ottawa Hospital,” said Mathieu LeBreton, digital experience lead, The Ottawa Hospital. “The opportunity to explore these technologies is well-timed, given the planning of the New Campus Development, a new hospital project in Ottawa. Proper identification of the problems we are trying to solve is imperative to ensure this is done responsibly and transparently.”
Deloitte is working with other hospitals and healthcare institutions to deploy digital agents. A patient-facing pilot with Ottawa Hospital is expected to go live by the end of the year.
Developers can get started by accessing the digital human NIM Agent Blueprint.