Large linguistic models. Does it imitate human writing or impose its own style?

Large linguistic models. Does it imitate human writing or impose its own style?

In an attempt to understand the nuances of writing style between humans and large linguistic models, on which generative AI technologies and tools are based, a team of researchers at Carnegie Mellon University conducted a recent study to assess the ability of these models to simulate human writing styles in different contexts.

Alex Reinhart, lead author of the study and associate professor in the Department of Statistics and Data Science at Carnegie Mellon University, said: “We humans have a unique ability to adapt our writing and speaking style to our surrounding contexts and situations, as we move seamlessly between formality and simplicity, and we are good at using a variety of methods to suit each situation.

So the question asked by the researchers was: do the generative AI models that we currently use at work and in personal life have the same adaptability?

Distinctive and unadaptable style:

The study revealed that large language models, such as ChatGPT and Llama, adopt a fixed and distinctive writing style and cannot fully adapt to different contexts as humans do. Reinhart emphasises that this study is the first of its kind that has been able to quantify this difference, a feat that has not been achieved before.

Methodology of study and discovery of differences:

Within the framework of the study, the researchers used large linguistic models to generate texts for various types of writing, such as TV show scripts and academic articles, and using an analytical tool developed by David West Brown, assistant professor in the English department at Carnegie Mellon University and co-author of the study, they were able to analyse the grammatical, lexical, and stylistic features of the generated texts and compare them with texts written by humans. The results revealed significant differences between texts produced by linguistic models and those written by humans.

These differences were most pronounced in modified instruction-tracking models—such as ChatGPT—that undergo additional training to answer questions and follow instructions, suggesting that the process of tuning with instructions may affect the ability of models to simulate the changing human style.

It has been shown that large linguistic models overuse the sentences of the present tense, as these sentences appear in their texts at a rate of two to five times higher than what we find in human writings.

In addition, the study revealed a noticeable tendency among models to use nominalizations—the process of converting verbs, adjectives, or other parts of speech into nouns—as these formulas appear at a rate ranging from 1.5 to twice the rate used by humans, which affects their writing style, as these linguistic structures contribute to making the text more informative and less smooth.

The study also showed a significant decrease in the use of the passive voice without a subject in the GPT-4o model, as it is half of what is typical for human writing.

These results reflect that large linguistic models have been designed and trained to follow an informative and name-packed writing style, making them less flexible in imitating the diverse writing styles created by humans

Large linguistic models are characterised by a linguistic footprint:

The study also revealed the presence of a distinctive linguistic imprint in the modified models of following instructions, manifested in the use of some words with a noticeable frequency compared to texts produced by humans in the same literary context; for example, the ChatGPT versions used the words camaraderie and tapestry with a frequency of up to 150 times compared to human writing.

Similarly, variants of the Llama model showed a marked preference for the word (anxiety) unease, with a frequency of 60 to 100 times that typical for human texts.

In addition, the study revealed common language preferences among the large language models, as both the ChatGPT and Llama models showed a strong tendency to use the words palpable (concrete) and intricate (complex).

What are the implications for education and writing?

“There was a lot of anxiety among teachers, and I thought as a person working in data science within the English department that this is not what writers actually do; writing is not just a one-time production of a text but an iterative process that includes constant revisions and revisions, Brown explained. In this context, Brown posed a central question: Can these models produce a one-time text that looks convincing?

Brown stressed that the main goal of the study is to make the public aware of the conditions under which large language models can be used appropriately. He explained that accuracy is the most important in some cases, such as taking doctor's notes, where the style does not matter as much as the correctness of the information.

In contrast, Brown stressed the importance of style and excellence in other writings, such as job application letters, and stressed the need for teachers, writers, and translators to be aware of the characteristics of large language models and their disadvantages.

For his part, Alex Reinhart expressed his concern about the students' use of large linguistic models in completing school assignments and refused to compare these models with calculators, stressing that they do not produce the same results as humans and explaining that the calculator performs the same calculations as humans but does not make mistakes, as these models produce texts different from those written by humans.

Future prospects:

The researchers noted the urgent need for further studies to understand the effect of instruction tuning on large language models. In this context, PhD student Ben Marki is leading a research project aimed at assessing the ability of these models to evaluate human writing, such as student essays, and to determine the consistency of their assessments.

This study reveals the limitations of large linguistic models in simulating human writing and raises questions about their use in various fields. While these models can be useful tools, we must be aware of their characteristics and limitations and use them with caution and awareness. especially in contexts that require creativity and flexibility of style that are characteristic of human beings