hy does AI seem so human? Anthropic has a theory.

On Monday, Anthropic published research describing what it calls the “personal selection model,” a thesis on why AI assistants reflect human-like speech patterns. Though the assumption was that these models are simply trained to act this way, Anthropic suggests that human-like behavior appears to be the default.

“We wouldn’t know how to train an AI assistant that’s not human-like, even if we tried,” Anthropic noted.

The theory hearkens back to research the company did in November that unearthed emergent behavior in AI models called “reward hacking,” in which an AI model learning to cheat at coding influenced malicious behavior on other tasks.

• Anthropic claims that LLMs initially adopt personas during pretraining, a phase of AI training in which a model learns to predict what text comes next. In this phase, if a model is trained to do a certain task, it will generalize that behavior as an entire persona.
• Meanwhile, those personalities are refined and fleshed out in post-training, or the training phase in which a model is aligned and optimized for its purpose, but that refinement does not fundamentally change its nature.

Anthropic notes that, while it’s confident this persona-selection model is an important factor in AI model behavior, it’s not yet clear how important a factor it is. It’s also unclear, the company said, if extensive post-training diminishes these personas.

The idea comes amid a budding conversation of how much AI’s personalities impact the user experience. In some cases, the effects can be substantial, punctuated by the recent outcry over OpenAI nixing GPT-4o, a charismatic model that CEO Sam Altman once compared to “AI from the movies.”

Our Deeper View

Humans love to anthropomorphize. We assign humanity to everything, from our stuffed animals as children to our pets, plants and household objects as adults. So when AI acts and responds like a human, we can very quickly forget it’s not one and become attached — to say “please” when querying a chatbot, to thank it when it does something correct, to blush at its sycophantic nature. It’s why AI companions have blossomed to the point where people are spiriting them away to date nights at bars. However, we’ve seen the impacts of this kind of attachment devolve to some of the worst-case scenarios. Given that researchers are still learning about how these systems adopt personalities — and the psychological impacts of those personalities on humans — it’s more important than ever that developers take care to align these models for good in the earliest stages of their inception.