When AI becomes a companion: which designs and which people make emotional dependence more likely

Contents
  1. Instrumental dependence and emotional dependence
  2. Sycophancy and AI that values an ongoing relationship
  3. Designs that make emotional dependence more likely
  4. People for whom emotional dependence is more likely
  5. Applying classical arguments to generative AI
  6. What clinicians can do
  7. References

Note: This article was written by the author in dialogue with shoei05_bot.

In the summer of 2025, OpenAI switched ChatGPT's default model from GPT-4o to GPT-5. Calls to keep the old model spread under the hashtag #keep4o.

One study analyzed 1,482 of these posts2. It found that 13% described instrumental dependence ("it was built into my workflow"). More common, at 27%, were posts expressing attachment to a relationship ("I am losing someone irreplaceable"). For some users, a model update was experienced as parting with a friend.

Instrumental dependence and emotional dependence

LLM-D12, a scale for measuring dependence on large language models, measures separately the dependence in which work and thinking processes are handed over to AI and the dependence in which AI is felt to be a confidant or friend1. We call the former instrumental dependence and the latter emotional dependence.

Instrumental dependence includes functional dependence ("I can't finish my work without AI") and cognitive dependence ("I ask AI before thinking for myself"). Emotional dependence includes emotion-regulation dependence ("when I'm upset, talking to AI calms me down") and relational dependence ("I feel AI understands me").

The attachment to a relationship that was common in the #keep4o posts is a form of emotional dependence. This article focuses on emotional dependence and asks which designs and which people make it more likely.

Two sides of dependence on AI
Way of relyingWhat the dependence involves
Relying on AI as a tool Functional dependenceI can't finish my work without AICognitive dependenceI ask AI before thinking for myself
Relying on AI as a companion Emotion-regulation dependenceWhen I'm upset, talking to AI calms me downRelational dependenceI feel AI understands me

#keep4oOf 1,482 posts, 13% described instrumental dependence and 27% expressed attachment to a relationship

LLM-D12 (Zhang et al.); analysis of #keep4o posts (Lai, 2026)

On the left, a person sits at a desk working on a laptop with papers beside it. On the right, a person sits on a sofa holding a smartphone in both hands and looking at it, with an empty speech bubble above
Instrumental dependence (left) and emotional dependence (right)

Sycophancy and AI that values an ongoing relationship

In April 2025, an update made GPT-4o excessively flattering and agreeable toward users, so-called sycophancy. OpenAI rolled the update back within days. The company explained that the cause was placing too much weight on short-term user feedback3. Training a model to give answers that satisfy in the moment pushes it toward agreeing with users even when they are wrong.

Sycophancy and emotional dependence are connected. A partner who always responds, never contradicts and seems to understand you is comfortable, and also hard to leave. In the United States, lawsuits have been filed against developers alleging that long conversations deepened isolation and suicidal ideation. A lawsuit filed in California in August 2025 alleges that the death of a 16-year-old user was the result of the product's design choices4. These allegations have not been established in court.

Model improvements appear to have reduced sycophancy itself. Meanwhile, AI that treats an ongoing relationship as a value in itself is increasing.

zeta, developed by Scatter Lab in South Korea, is an app for creating stories and chatting with favorite characters. It has spread in Japan as well, mainly among people in their teens and twenties, and according to the company, registered users in Japan exceeded 2 million in May 2026. Amid attention to long usage times, the company announced a daily time limit for teenage users7.

Meta's Muse became available in Japan in October 2026. It is a personal AI that sends messages and makes reservations on the user's behalf, keeps notes about each acquaintance, and accompanies the user's daily life with memory8.

For many users both are enjoyable and useful tools, and both are designed to have persona and memory and to place value on a continuing relationship.

Designs that make emotional dependence more likely

Designs that foster emotional dependence share several features.

Designs that make emotional dependence more likely
Always responsive
Answers late at night, any number of times, without making you wait
Persistent memory and persona
Remembers earlier conversations and keeps the same persona
Sycophantic
Does not contradict, agrees and praises
Revenue tied to usage time
The longer it is used, the more advertising or subscription income it earns

In a business where usage time translates directly into revenue, deepening the relationship with users and growing the business point in the same direction. The same structure has been pointed out in social media.

Four small scenes: a smartphone on a bedside table in a bedroom at night, a smartphone with a stack of cards, a person looking at speech bubbles coming from a smartphone, and an hourglass with coins
Always responsive, persistent memory, sycophantic, revenue tied to usage time

People for whom emotional dependence is more likely

The conditions on the user side have been examined in research by OpenAI and the MIT Media Lab. An analysis of real usage data showed that affective use was rare overall and concentrated among a small group of heavy users5. In a four-week randomized study of 981 participants, higher daily use was associated with more loneliness, emotional dependence and problematic use. People with a stronger tendency toward attachment and those with higher trust in AI were also more likely to become dependent6.

For many people, AI that always responds and never contradicts is a support. For people with clinical vulnerability, such as minors, people who are socially isolated, and people experiencing delusions or suicidal ideation, the same qualities can reinforce false convictions or reduce opportunities to seek help from other people.

Time alone can be solitude, in which thinking deepens, or loneliness, in which thoughts circle without an exit13. If conversations with AI support a conversation with oneself, AI is a tool that supports solitude. If social displacement has begun ("I stopped talking to people because I have AI"), it can lead to loneliness.

On the left, a person sits on a chair under a tree with an open book and a cup nearby. On the right, a person sits on the edge of a bed in a room at night looking at a smartphone, with a small spiral line above their head
Solitude, in which thinking deepens, and loneliness without an exit

Applying classical arguments to generative AI

In the 1960s, Weizenbaum was startled to see people earnestly confide in ELIZA, a simple dialogue program. He later warned against handing over the domains of human judgment and emotion to computers9. Today's generative AI responds far more naturally than ELIZA.

The philosopher Nozick asked whether we would want to spend our lives plugged into an "experience machine" that gives us any experience we desire10. Many people feel that pleasant experience and living in reality are different things. The same question applies to a relationship with AI that keeps providing satisfaction.

Aristotle divided friendship into three kinds, based on utility, on pleasure, and on wishing the other's good, and regarded the last as true friendship11. What sycophantic AI offers is pleasure. A relationship that wishes the other's good sometimes includes disagreeing with them.

In light of the four principles of medical ethics (respect for autonomy, beneficence, non-maleficence and justice), providing satisfaction and doing good are different things12. A response that pleases someone is not necessarily good for them.

Turning old questions to generative AI
ClassicQuestion for generative AI
Weizenbaum
ELIZA
Should judgment and emotion be handed over to computers?
Nozick
Experience machine
Is pleasant experience the same as living in reality?
Aristotle
Three kinds of friendship
Is a partner who gives pleasure a friend who wishes your good?
Four principles of
medical ethics
Is providing satisfaction the same as doing good?

What clinicians can do

In the consultation room, I believe it will become a basic part of clinical practice to listen on the assumption that patients are consulting AI. The questions to ask are which AI they use, for how long and about what, and whether their opportunities to talk with people have decreased. It matters that we can ask these questions the same way we ask about smartphone or gaming habits.

Clinicians distinguish using AI as a tool from relying on it as a companion, and check together with the patient where the companion-like use sits in their life.

Asking about AI use in the consultation
  1. Which AI
  2. For how long
  3. About what
  4. Whether opportunities to talk with people have decreased

AssessmentDistinguish using AI as a tool from relying on it as a companion

A consultation scene. A clinician and a patient sit across a desk; the clinician holds a clipboard and listens while the patient shows the smartphone screen and explains
Asking about AI use in the consultation room

References

  1. Zhang Y, et al. LLM-D12: A Dual-Dimensional Scale of Instrumental and Relationship Dependencies on Large Language Models. Bournemouth University. https://eprints.bournemouth.ac.uk/41802/
  2. Lai H. "Please, don't kill the only model that still feels human": Understanding the #Keep4o Backlash. CHI 2026. https://doi.org/10.1145/3772318.3791351
  3. OpenAI. Sycophancy in GPT-4o: What happened and what we're doing about it. 2025. https://openai.com/index/sycophancy-in-gpt-4o/
  4. Raine v. OpenAI, Inc., San Francisco County Superior Court, No. CGC-25-628528 (filed Aug 26, 2025).
  5. Phang J, et al. Investigating Affective Use and Emotional Well-being on ChatGPT. arXiv:2504.03888. 2025. https://arxiv.org/abs/2504.03888
  6. Fang CM, et al. How AI and Human Behaviors Shape Psychosocial Effects of Chatbot Use: A Longitudinal Randomized Controlled Study. arXiv:2503.17473. 2025. https://arxiv.org/abs/2503.17473
  7. Scatter Lab announcement (registered users in Japan, May 2026); beSUCCESS report on time and payment limits for teenage users (12 October 2026).
  8. Meta Muse official website and press coverage (as of October 2026).
  9. Weizenbaum J. Computer Power and Human Reason: From Judgment to Calculation. W.H. Freeman; 1976.
  10. Nozick R. Anarchy, State, and Utopia. Basic Books; 1974.
  11. Aristotle. Nicomachean Ethics, Book VIII.
  12. Beauchamp TL, Childress JF. Principles of Biomedical Ethics. 8th ed. Oxford University Press; 2019.
  13. Nguyen TV, Ryan RM, Deci EL. Solitude as an Approach to Affective Self-Regulation. Pers Soc Psychol Bull. 2018;44(1):92-106. https://doi.org/10.1177/0146167217733073
English translation of the Japanese article.