Doctors say that cancer education equipment created from an AI

• According to a new study.
• Materials have shown improved readability and loyal translations into several languages.
• Clinicians may be able to take advantage of this technology to extend the health literacy of patients and reduce the workload, but serious questions remain on the precision of the materials generated by the AI.

Can AI help improve patient's health literacy? Yes – a new study suggests.

The tools of artificial intelligence, including models of large languages ​​(LLM) and generative AI, continue to make breakthroughs in health care. A promising application is patient education, which could benefit both clinicians and patients.

In a new study published this month in Plos aThe researchers found that the LLMs formed on directives in urological oncology could produce educational material (PEMS) of patients as precise and complete as the source material.

Improving readability scores indicated that the PEM generated by the LLM were also easier to understand for patients, and they could be translated relatively precisely in other languages.

The PEM generated by the LLM could extend health literacy and reduce the workload for health care organizations that create and translate these critical documents, suggest the authors of the study.

“It is logical that (an LLM) should be able to correspond well to these materials in precision and exhaustiveness,” said Jonathan H. ChenMD, PHD, assistant professor of biomedical computer science and science of biomedical data at Stanford Medicine, which was not involved in research. “Since it is a large language model, what is particularly good is to rework the writing. The translation between languages ​​is already surprisingly good, but imperfect. ”

Chen notes that the use of AI to help the translation is logical, because clinicians are already engaged in a tacit form of “translation” every day when they explain complex medical details and jargon to patients in clear language.

However, the authors of Chen and the authors of the study note the reservations on the overall precision of the materials generated by LLM, and neither of them suggests that they are ready for traditional use. Instead, the authors offer a future workflow model in which these documents are examined by human experts before publication and dissemination.

Such a workflow demonstrates “a good combination of computer and human attention to produce reliable information,” said Chen.

The main objective of the researchers was twofold: assessing the accuracy of a PEM generated by the LLM in English and assessing its translations in other languages.

To achieve this, they have formed GPT-4 (an LLM developed by Openai, the company behind Chatgpt) to extract information on the guidelines from European Urology Association (Water) On localized and metastatic cancers.

The LLM then produced an English version of the PEMS and translated them into five European languages: Spanish, Dutch, French, German and Italian. In total, 48 PEMs were generated for four localized cancers and four metastatic cancers.

A group of 32 certified urologists of the board of directors of the young academic network of academics assessed the PEMS for accuracy, completeness and clarity. The evaluation was blinded, so the experts did not know if a PEM was an original water equipment or generated by LLM.

The researchers evaluated several criteria comparing the PEM generated by LLM with source materials. The most obvious discovery was the LLM speed to generate equipment: the average time to create a PEM was less than a minute – only 52 seconds.

The three areas of writing quality – precision, completeness and clarity – were tied with source material. Meanwhile, the readability, measured by two distinct measures, has been considerably improved. For example, water materials had a 11th year reading level, while the generated version required only a sixth year level.

“AI models are formed on a more natural language of daily life, which is certainly easier to understand. Sanjay AnejaMD, scientific doctor and researcher at the Yale Cancer Center and assistant professor of therapeutic radiology at the Yale School of Medicine who was not involved in research.

The members of the Yau network with a native mastery in each language then evaluated the translated versions. They evaluated the translations for loyalty to original English and for more clarity.

In the five languages, 77.5% of translations were noted faithful and 67.5% were clear.

Translation of critical material in additional languages ​​offers a major opportunity to expand patient knowledge, as study authors note that certain water materials are still only available in a few European languages.

Only English and German had original PEMs for the four cancer types in the study. The Spaniard had two original PEM available, while the Dutchman, the Italian and the French one had each one.

Although promising, the materials generated by LLM are not yet precise enough or reliable to be autonomous.

“Human surveillance is always required in this approach to ensure ethical use and responsible dissemination to maintain the integrity of production,” wrote the authors of the study.

However, their proposed workflow – combining a generative AI with human expertise – could use the advantages of this technology while maintaining the great precision required in health care. This could lead to less workload for experts involved in maintaining the PEMS precise and up to date, and translated them. For patients, more widely accessible PEMs could mean improving literacy and health results.

Aneja agrees with this evaluation.

“I think the most effective model for AI is not the replacement of clinicians, but rather” increased intelligence “where clinicians exploit AI as a tool to make their clinical practice more effective,” he said.

For the moment, however, there are still many risks involved.

It is common for the LLM inaccuracies And a disinformation, and even “hallucinous”, generating misleading and insensitive answers. Some research also suggests that AI models can give biased Health care treatment recommendations.

Until the AI ​​reaches a threshold recognized for medical precision – and perhaps even – human expertise must act as the final railing.

“Although a large part of the content generated by AI is in fact useful and precise in medical contexts, there are still enough risks for users (clinicians and patients) to be vigilant for very convincing disinformation,” said Chen.