Generative AI systems known as Large Language Models (LLMs) represent a potential hazard to the integrity of scientific knowledge. This risk is primarily due to the occurrence of fabricated responses that these models can produce. To safeguard the integrity of scientific information, a group of esteemed researchers from the Oxford Internet Institute, including Brent Mittelstadt, Chris Russell, and Sandra Wachter, suggest tighter controls on LLMs. Their work has been featured in the journal Nature Human Behaviour.
The trio elucidates that LLMs are engineered to deliver convincing and useful answers but lack mechanisms to ensure factual correctness. These models are often fed from enormous compilations of internet-based text, which not infrequently include inaccuracies, subjective viewpoints, or imaginative content.
Mittelstadt remarks upon the human tendency to ascribe human traits to LLMs, trusting their information as we would a fellow human. This anthropomorphic view is compounded by LLMs’ facility in mimicking conversational partners, responding readily to queries with seemingly authoritative text, despite potentially lacking a factual foundation.
To preempt the misapplication of LLMs in scholarly and pedagogical contexts, the scholars recommend establishing clear and conscientious use guidelines. They propose guiding users to frame requests that draw upon vetted, reliable data.
Wachter emphasizes that the manner in which LLMs are utilized is crucial, especially where unquestionable factual accuracy is imperative, such as in the realm of science. Russell suggests a contemplative approach to LLMs, pondering the implications of their capabilities beyond their immediate technological allure.
In practice, LLMs often function as databases, supplying information upon request. This puts users at the mercy of both the model’s regurgitation of any inaccurate data it was trained on and the generation of outright fabrications—misinformation the LLM creates absent in its training materials.
To address this, the experts propose using LLMs as tools for data translation on a case-by-case basis rather than as sources of truth. Instead of relying on the LLM for information, users should guide it to reframe or reformulate accurate information that the user provides. For instance, using the model to expand concise notes into a detailed summary or to build scripts that convert scientific figures into visual representations.
By using LLMs in this way, it becomes simpler to verify that the resulting output is true to the provided facts. While recognizing LLMs’ potential to facilitate scientific tasks, the authors maintain that vigilant examination of their outputs is essential to maintaining rigorous scientific standards.
Brent Mittelstadt, an authoritative research leader at the Oxford Internet Institute, calls for the use of LLMs as tools for precision translation in scientific applications, underscoring the crucial nature of their responsible deployment.