The dialogue between doctors and patients is the key to diagnosing the disease and establishing an effective treatment plan. However, in reality, not everyone has access to abundant medical resources and in-depth consultations with doctors.
To solve this dilemma,GoogleResearchers have developed aMedical consultationThe large language model AMIE (Articulate Medical Intelligence Explorer).
AMIE utilizes a novel self-dialogue simulation environment combined with an automated feedback mechanism to enable learning across different disease conditions, specialty areas, and contexts.
The researchers evaluated AMIE alongside 20 primary care physicians in 149 clinical cases and found that AMIE outperformed the physicians in multiple areas, including diagnostic accuracy and communication skills.
Paper address: https://arxiv.org/abs/2401.05654
Simulate conversation environment
AMIE was developed based on Google's PaLM2 model. In order to adapt AMIE to doctors and cases of different specialties, researchers used the original "simulated patient dialogue environment" for training and fine-tuning.
It can automatically generate various case data based on online search results, and AMIE can continuously improve its own abilities by taking turns to learn and talk with simulated patients..
The R&D team randomly selected disease conditions from three medical databases and generated various case descriptions through online search results. AMIE then played the role of patient and doctor respectively, and conducted a dialogue under the supervision of a third-party evaluation mechanism.
There are three roles in the environment: patient template agent, doctor template agent and evaluation template agent.The patient agent answers questions based on the case description provided; the doctor agent asks questions to analyze the condition; the evaluation agent gives feedback based on the quality of the conversation, helping the doctor agent to gradually improve their communication performance.
This simulation environment can automatically generate a large number of large-scale medical conversations, making up for the lack of real doctor-patient communication data and effectively expanding the learning scope of AMIE.
Self-learning loop
The researchers designed an inner and outer loop mechanism to help AMIE achieve self-learning. In the inner loop, AMIE will continuously improve the effect of simulated dialogue based on evaluation feedback; in the outer loop, simulated dialogues are collected to enhance AMIE's subsequent training and establish a self-refinement loop.
As training continues, AMIE's interactive capabilities will continue to improve; at the same time, through external loop learning, its medical knowledge will continue to expand and adapt to more medical scenarios.
Chain reasoning strategy
In real medical conversations, AMIE adopted a three-step chain reasoning strategy.FirstThe first step is to summarize the patient's symptoms and propose a preliminary diagnostic hypothesis based on the conversation history; the second step is to customize the response plan and the next consultation goal on this basis, aiming to further determine the diagnosis;
The third step is to check whether the response is smooth and accurate, avoid repeated questions and wrong information, and make necessary revisions and corrections. The researchers believe that AMIE can achieve higher diagnostic accuracy than a single response by continuously optimizing through chain thinking.
Test Data
To evaluate the performance of AMIE, the researchers designed and conducted a validated remote objective structured clinical examination in which AMIE interacted with validated simulated patients or primary care physicians via a text interface.
A total of 149 clinical cases were involved, 20 primary care physicians were compared with AMIE, and evaluated by professional physicians and patients.
The results show thatAMIE outperformed primary care physicians in diagnostic accuracy and 28 of 32 assessment dimensions, as assessed by specialists; According to the patients' evaluation, AMIE showed better diagnostic results in 24 out of 26 evaluation dimensions.