Relationships Do Not Scale

On the limits of AI’s capacity for human relationships

I would like to continue the reflections I began last year on how AI is changing the landscape of psychotherapy and human relationships more broadly. Since then, my predictions and views have shifted. This is primarily due to new clinical experience, both in my own practice and in the work of colleagues. I also cannot ignore what I encounter in everyday life: not only stories about strangers whose lives have been profoundly changed by AI, but also situations I see within my own close circle.

An intriguing essay by Professor Ivan Yamshchikov, which I came across recently, helped me give these observations a clearer shape.

The essay describes an interesting paradox: in modern society, the number of “shamans” — teachers, doctors, psychotherapists, coaches and so on — per capita is growing: 1:90 compared with 1:500 in prehistoric tribes. And yet demand remains chronically unmet, while in indigenous tribes the shaman seems to meet people’s needs reasonably well.

One of the important questions raised in the essay is whether AI could solve this problem and help meet the demand for “mentorship” in the broadest sense. The author sees AI sycophancy as one of the key problems: a kind of ingratiating behaviour in which, instead of giving an honest answer, AI tries to flatter.

I see another problem, one that seems to me even more fundamental than sycophancy. It lies in the fact that current AI architectures — throughout, I mean specifically LLMs — are incapable of possessing something that is constitutive of human relationships.

---

Two Dimensions: Expert and Relational

Let me begin by provisionally distinguishing two dimensions in which all helping professions operate.

The first I’ll call the expert dimension. This is when, in response to a description of a situation and a set of goals, someone offers the most effective way to achieve them. For example, a lawyer, having heard the situation, may refer to laws and legal procedures. A doctor may take a patient’s history as input and refer to diagnostic guidelines in order to prescribe treatment.

The modern world, oriented toward scalability, places its emphasis on precisely this component: reducing everything to standardised procedures verified by experts. This is where the tendency toward an increasing number of helping professionals per capita has its roots: for each new problem domain, an ever narrower specialisation emerges, attempting to cover that domain as fully and as systematically as possible. The underlying promise is that by narrowing each specialisation, we can collectively cover an ever wider spectrum of human problems.

The second is the relational dimension. This is the part that allows a person to feel the presence of another human being alongside them. It is from this dimension that motivation, sympathy, care, expressions of concern, and other feelings arise. This is not only about psychotherapy. A client who comes to a lawyer brings, among other things, emotions connected with a sense of injustice. A pupil may feel fear or overconfidence, both of which a teacher must take into account. This is difficult to package into a manual or a self-help book. Relationships do not scale well.

At the moment, AI appears to be an ideal solution: it scales expertise remarkably well, possessing an enormous volume of information and the ability to work with it.

I see this in practice. My clients are increasingly using AI alongside individual therapy. And almost always, they come back with answers from the expert dimension. AI can lay things out clearly, suggest practical solutions, and refer to widely accepted approaches. And it is wonderful that this kind of help exists. But you cannot get very along the expert dimension alone.

It is rather like watering a plant in sandy soil. Some of the water will, of course, settle around the roots, but most of it will simply pass straight through. This is why finding “root causes”, receiving practical recommendations, and obtaining other analytical answers — even insights — ultimately account for only the smaller share of the therapeutic effect. Most of the work takes place in the relational dimension¹.

So there is a temptation to assume that AI can solve the problem of scaling relationships as well: that it can form relationships with people without the usual human constraints. But there is a problem.

Lived Presence and Shared Experience

To clarify this problem, we need to look at the component around which human relationships are formed: the capacity to be present with another person and, in some measure, to share in their experience. This occurs when one person recognises the situation another person is in, and the other person understands that their situation has been recognised. Through this, we become aware of another person’s presence beside us and feel less alone.

The obvious case is when both people are in the situation together. For example, both are about to sit the same exam; or both have just won a match; or both have been caught in the rain.

But it is not necessary to be in the situation at the same time as the other person. One can recognise another person’s situation through one’s own lived experience. When one person shares an experience with another, the other person, to some extent, recognises that experience and “co-experiences” it.

This is not simply cognitive recognition, but recognition at the level of the body — or, if you like, the nervous system. If I tell you that I have just taken off a tight shoe, you will, to one degree or another, reproduce that sensation, recognise it, and partially feel the relief — rather than merely infer it intellectually.

But pleasant experiences are the easy case. It becomes more difficult when you share something painful with another person, because they have, broadly speaking, two options. The first is to bear the feelings evoked by what they have heard and remain with them. The second is to distance themselves from those feelings and not live through them. For example, if I say that I have burned myself, you may fully feel into that experience and show real sympathy, giving me the sense that I am not alone. The same can apply to more complex experiences, such as grief after the loss of someone close.

So, when you share your experience with another person, there is always a risk that they might find it overwhelming. There is no escaping this risk — neither in therapy nor in relationships with loved ones. A therapist might not be able to handle it and emotionally check out. A loved one might not be able to handle it and leave.

When people cannot bear another person’s intense feelings and avoid them, this is often not a conscious choice, but an unconscious reaction. But when they stay, this is almost always a choice. This choice comes at a price: living through something unpleasant. They have, as Nassim Taleb would put it, skin in the game. The fact that the other chooses to stay, at the cost of their own comfort (even in exchange for something of higher value, like a close relationship), is what creates the capacity for lived presence. And this capacity is essentially the basis for building relationships.

Three Limits of Simulated Presence

The problem with AI is that it cannot fully develop this capacity for lived presence — it simply cannot choose to remain with another person’s difficult experience. I see three reasons for this.

First, AI does not make a choice; it acts according to a predetermined procedure for selecting the next step.

Second, even if we accept the determinist view that human beings have no choice either, AI still has no skin in the game: it pays no price for its choice, moral or material. It does not shorten its lifespan, nor does it spend scarce resources. The only “interest” AI can be said to have is optimisation toward an acceptable answer — which makes sycophancy the natural long-term shape of the interaction.

Last but not least, the model cannot know what it is like to experience what a human being experiences. To know not in the sense of a textual description, but to know in the sense of bodily experience. This is precisely what Nagel had in mind when he asked whether we can truly know what it is like to be a bat. Suppose we are maximally charitable and grant that models may have something analogous to consciousness, or that qualia may be present in their experience. Even with this assumption, we can say with certainty that they do not share with us the biological basis that would allow them to have experiences 100% similar to ours.

One could object: even between human beings this basis is never complete. A man cannot know childbirth from the inside; someone who has never known depression cannot fully understand it. This is true. But there is a difference between partial overlap, which exists between any two human beings, and its complete absence, as in the case of a model. The point is not a perfect match between experiences, but the presence of at least some shared bodily ground. It is this ground that gives us an experience we can draw on in order to relate to the other.

This is perhaps clearest in the case of shame. If you share with AI something you feel ashamed of — for example, that you betrayed someone — it may lift part of the burden by seeming to accept you as you are. But you are unlikely to believe AI if it says, “I know what that is like. I have betrayed someone too.” If another human being says this, you do not merely feel that you have not been judged. You feel less alone, because someone else knows what it is like to live through something similar. They share enough of that experience for you to feel less alone in it².

Because of these three factors, the model, no matter how much it may appear to want to, cannot choose to remain with human experience and bear it.

As an additional illustration of this point, I would like to consider an intermediate case: relationships with animals. Animals differ from us in many ways, but they still have much more in common with us than AI does. Dogs, for example, have a clearly recognisable therapeutic effect — and each of the three factors is present in them to some degree.

They possess a certain degree of freedom of choice: they can leave, turn away, or refuse to come closer. They pay their own price: there is often something nearby that would bring them more pleasure than staying with a human being’s distress, and by remaining, they give it up. Finally, although anatomically we are far apart, we have much in common — from the nervous system and its chemistry to the eyes in which we see our own feelings reflected — while they see something of theirs in ours.

This shared basis makes affect mutually readable: not simply “we both feel something”, but “we can recognise something in one another”. Of course, this is not relationship in the full sense in which human beings offer it. But nor is it AI, where all three factors tend towards zero.

The Price of Real Presence

None of this means there are no relationships between humans and AI at all. They are quite real, but they are one-sided: the human being invests; the AI responds, but does not truly participate. Structurally, this is closer to attachment to a machine or a houseplant than to a relationship between two people — except that AI creates more complex feedback loops, which makes the attachment more compelling.

Nor does this negate AI’s growing capacity to imitate such relationships. As I wrote in last year’s article, the more data — especially multimodal data such as video and audio — is added to training inputs, the better AI will be able to imitate relationships. But this imitation will have a limit of presence, which I have outlined in the three factors above. In some contexts, that limit will be so distant that it will not affect the value of the help provided. But in others, especially in psychotherapy, even the anticipation of this limit may quickly reduce the value of the help to nothing.

Seen from this angle, here is how AI will affect the future market for the helping professions — and psychotherapy in particular. Demand in areas that rely on expertise will be met more and more effectively. The same will happen in areas that can be covered by an “imitation” of relationship, whether through sycophancy or something else. All of this can be scaled.

But in domains where genuine human-to-human relationship matters, current architectures will not replace humans any time soon.

AI one day acquires all three — choice, a price to pay for that choice, and the basis for living through human experience — and the question of relationship is solved, a more difficult question will arise: how, in principle, would such an AI differ from a human being?

And more importantly, would it be possible to use such AI to eliminate the deficit of relationship? After all, if an algorithm gains the freedom to choose whether it wants to live through another’s pain, it will cease to be a scalable instrument: it will have something to lose.

The price of real presence cannot be reduced to zero. Which means that we return to where we began: real relationships do not scale.

1. This is supported by meta-analyses, the best-known being Wampold’s: most of the differences in therapy outcomes are explained not by method, but by common, essentially relational factors — alliance, empathy, expectations, and the effect of the therapist themselves.

2. This, incidentally, is one of the reasons why therapy groups can be at least as effective as individual therapy: there are more participants, and therefore simply more chances that you will recognise your own experience in someone else’s.