There is a form of pressure that operates in every close relationship, continuously, sometimes invisibly, and the people applying it are only sometimes fully conscious that they are doing so. It is behavior shaping, the operating system of intimate life, and once you can see it operating in the home, you can see it operating in the boardroom, in the political campaign, in the pulpit, in the algorithm, and in the chatbot that is helping you write your next email.
Both the shaping and the being shaped are ancient survival machinery. To stay inside the group was to live. To drift outside it was to die. The firmware on both sides of behavior shaping evolved together, across hundreds of thousands of years. Neither side is defective; both are adaptive and work exactly as designed. The trouble is that the equipment was designed for a village of forty people, and it is now running in a civilization of eight billion connected by glass screens.
The scholar worth knowing
I have learned a great deal from the evolutionary psychologist Diana Fleischman. Fleischman’s research describes disgust and disapproval as the affective core of human social regulation. Disgust is older than language. If I’ve understood her correctly, disgust evolved first as a defense against contaminated food and diseased bodies, and it was then recruited (as evolution so often recruits older machinery for newer purposes) as a mechanism for marking socially unacceptable behavior. The micro-expression of disapproval on another person’s face — the slight tightening around the mouth, the small withdrawal of eye contact, the cooling of tone that you felt rather than heard — is disgust running in its social register. It is the signal that says that you have drifted outside the acceptable, and warmth is being withdrawn until you drift back.
Children calibrate against this signal with extraordinary sensitivity because the developmental cost of failing to read it is exclusion, and exclusion in the ancestral environment was death. By the time any of us is an adult, the calibration has become deeply entrenched. The child has learned to watch a parent’s face for the first flicker of disapproval, and the adult knows when they have said the wrong thing at the dinner table, in the meeting, or in the group text. The disgust response in someone else’s face reaches us before the conscious mind has even finished parsing the sentence that triggered it.
Where the sophisticated version came from
Fleischman’s work sits inside a broader tradition in evolutionary psychology that traces the origins of sophisticated human influence to a specific asymmetry. The biologist Robert Trivers laid out the logic in 1972. In any species where one sex invests more per offspring than the other — in humans, overwhelmingly the female, through gestation, lactation, and the prolonged vulnerability of the child — the higher-investing sex faces stronger selection pressure for caution in mate choice, for relational vigilance, and for the development of indirect rather than direct competitive strategies. Physical confrontation was closed to that sex as a primary tool, monopolized by the lower-investing party with greater upper-body strength. Something else had to evolve in its place.
What evolved was the influence architecture Fleischman and others describe. Emotional attunement, the reading of subtle signals, the management of warmth and its withdrawal, the construction and control of narratives about oneself and others, coalitional alliance-building, reputation as a social weapon, the fine-grained calibration of approval and disapproval. This toolkit emerged first in its most refined form in the female repertoire for reasons that are genuinely not moral but mechanical. The mother-child dyad, in particular, is the laboratory where the sophisticated version of the toolkit was honed. A mother cannot physically force a toddler to do anything useful. She can only shape. The entire developmental architecture of the child is calibrated to be shapeable by exactly the signals the mother is equipped to send.
That is where the mechanism came from, but the evolutionary origin is not the story. The story is what the mechanism became.
The universal activity
Behavior shaping is not a female activity. It is a human activity, running in every direction, at every scale, through every channel, at every moment of social life. Men run it. Women run it. Children run it on their parents. Parents run it on children. Friends run it on friends. Colleagues run it on colleagues. Strangers run it on strangers in the first thirty seconds of meeting. The toolkit generalized from its evolutionary origins because it worked, and because language — the uniquely human capacity that let the toolkit extend beyond the reach of the face and the voice and the immediate relationship — made it almost infinitely portable.
The mechanisms in all of these cases are the same: a gradient of warmth; approval given when the other person stays within the acceptable range; warmth withdrawn, subtly, below the level of what could be pointed to or named, when they drift outside it. Integrated over thousands of micro-interactions, this is what produces what a person will say, what they will think, and eventually what they will believe. It is the continuous, low-grade application of social pressure, operating at the visceral level.
Both of my parents ran these behavior-shaping mechanisms with considerable intensity, in different registers. Dad was a traditional narcissist — the kind whose internal sense of self required the world around them, including his children, to reflect back a particular image of his own importance. Mom was an emotion-seeking narcissist — the kind who required emotional support at all times and experienced a diversion of that focus as a kind of injury. I use the word narcissism without apology because it is the accurate word, and because when I describe my parents to peers of my generation, almost all of them describe their own parents in similar terms. It was not a private family pathology, more like a generational signature.
This is not a grievance; it is a case study. Every reader of this essay was shaped by some version of this mechanism, because every human child is. The particular pressures vary enormously across families and cultures, but the machinery is universal.
The internalizing of the shaper’s voice
One feature of the mechanism is worth pausing on, because it is the thing that most clearly demonstrates how thoroughly the machinery gets installed. The shaping does not require the shaper’s presence. Long after a parent is dead, or a relationship has ended, or a friend has moved across the country, the voice continues to operate inside the adaptive mind of the person who was shaped by them. You imagine their reaction. Your behavior adjusts. The voice is now in you.
This is not metaphorical. It is how the system works. During the developmental window, the adaptive mind absorbs the specific shapes of the shapers who mattered most, and it runs their simulated reactions forward in time as part of its own decision-making machinery. This is efficient, from an evolutionary standpoint. The child internalizes the group’s norms, carries them forward into adult life, and continues to be regulated by them even when the group is not present. It is also one of the deepest explanations for why most people, most of the time, behave in ways that would satisfy people who are no longer in their lives at all.
Outward from the family
The same mechanism runs in every other intimate relationship. The marriage in which one person’s moods quietly govern the room, while everyone else manages around them. The friendship that cooled after one of you expressed the wrong political opinion. The workplace team in which certain topics simply do not get raised because everyone has learned, without being told, what produces disapproval from whoever holds the unofficial social power. None of this is conspiratorial. None of it is even fully deliberate in the usual sense. It is the ancient machinery running its ancient program in environments that look nothing like those in which the program was written.
And then the mechanism scales.
From the village to the population
Language was the first amplifier. The human capacity for narrative meant that the influence architecture could also operate at a distance and across time. The orator shaped thousands in an afternoon. The lawyer shaped a jury. The preacher shaped a congregation, and across centuries and even entire civilizations. Writing extended the reach further. Print extended it further still. Cultures that have thought seriously about this have been wary of the combination of linguistic fluency and the behavior-shaping toolkit in a single talented person, because the combination is politically powerful.
In the early twentieth century, Edward Bernays made the mechanism explicit. He was Sigmund Freud’s nephew; he had access to his uncle’s framework for the unconscious, and he understood that the hidden machinery running below awareness could be deliberately engaged to shape behavior at scale. He called it public relations. In his 1928 book Propaganda, he wrote, with extraordinary frankness, that the intelligent manipulation of the organized habits and opinions of the masses was a necessary feature of democratic society, and that an invisible government of capable people should organize the world. Bernays was not describing this arrangement with regret. He was advocating for it. He believed that the democratic project required this kind of hidden steering.
Bernays opened the door. What he articulated in 1928, as a theory of how modern governance did and should work, became, over the following century, the operational premise of most of the apparatus that shapes public opinion. And in 2009, when Barack Obama appointed Cass Sunstein to head the Office of Information and Regulatory Affairs, the door Bernays had opened reached institutional expression. Sunstein was the co-author, with Richard Thaler, of Nudge — a book that argued explicitly that the role of government is to shape citizens’ behavior through careful architectural manipulation of their choice environments. Not by presenting reasoned arguments to people capable of evaluating them. By structuring the environment so that the desired behavior emerges without the citizen noticing, they have been steered toward it.
Nudge is not a fringe book. It is a governing philosophy, enacted at the highest level of American government and adopted across the OECD. The philosophical shift it represents is not a shift in technique. It is a shift in the theory of governance itself. Madison’s Federalist 10 assumes a deliberative citizenry. Sunstein’s nudge assumes a citizenry that is steered. The distance between those two assumptions is the distance between the founders’ understanding of democratic legitimacy and the one most contemporary governments actually operate under.
Polling is the scientific-credibility variant of the same move. Dressed in the language of measurement, presenting itself as simply discovering what people think, but actually a tool for constructing what people would think next: telling you what everyone else thinks, which is one of the most reliable predictors of what you will decide you think tomorrow.
And then the platforms. Sean Parker, one of Facebook’s founding presidents, described the design objective plainly in 2017. The question the builders were asking, he said, was how to consume as much of your time and attention as possible. The answer they found was a social-validation feedback loop, and Parker openly acknowledged that they knew they were exploiting a vulnerability in human psychology while building it. The vulnerability they exploited was the same one Fleischman’s work describes. The calibration of the subconscious mind to the gradient of social approval. The sensitivity to subtle signals of acceptance and withdrawal. The ancient toolkit, industrialized, now aimed at billions of humans simultaneously, individually targeted, continuously refined by the behavioral data users themselves generate with every tap.
What is happening
The machinery that evolved in the mother-child dyad, that was then generalized into the full human influence architecture, that was then extended by language and writing and print and broadcast, that was then articulated by Bernays as a governing principle and institutionalized by Sunstein as official doctrine, that now runs through every device in every pocket with precision Bernays could not have imagined — this machinery is the water we are swimming in. Most people cannot see it because it produces their own thoughts, feelings, and reactions. That is not a bug. That is the mechanism working exactly as it has always worked. The shaping is invisible because invisibility is how shaping works.
Back to the kitchen
But the machinery that matters most to any of us, still, is the version running closest to us. All of the industrial-scale shaping — the platforms, the polls, the nudges, the ads — operates on an adaptive mind that was originally calibrated by a small number of specific people, mostly during childhood, whose approval and disapproval taught the system what to want and what to avoid. The internalized voice is still there. The parent who has been dead for a decade is still in the room when you choose what to say. The friend whose slight disapproval of a view you once expressed is still shaping what you will and will not write around them. The partner whose mood you have been managing around for years.
The shaping runs both ways. We do it too. The slight change of tone. The withdrawal of warmth. We don’t see it as shaping. We say we’re reacting honestly to what seems obviously to require a reaction. The recognition is that both sides of the machinery are running in us at every moment. We are being shaped. We are doing the shaping. And the people closest to us are the most consequential operators of the mechanism on us, and we on them, because proximity amplifies the effect. The voices are still there, running inside our relationships and inside our heads, long after we left them.
There is no escape from being a human in a group, but we have the capacity to notice, sometimes, that the voice making the decision is not our own.
The Machinery Arrives in a New Form
Every previous amplifier of the behavior-shaping toolkit had a natural boundary, but it kept expanding. The orator could reach a crowd, but only one crowd at a time, and only for as long as his voice held out. The printing press could reach readers across distances the orator could not, but the same text went to everyone, and the author had no say in who was reading or how they were responding. Broadcast extended the reach further, to millions simultaneously, but the message remained one-to-many, untuned to the individual. Social media was the first real step-change in targeting, because the platforms could observe each user’s behavior and adjust what they were shown in near real time, individually. That was the innovation that made Facebook and its successors what they are. The toolkit was no longer being applied to generalized audiences; it was being applied to individuals, one at a time, at scale.
And now the boundary has expanded exponentially.
A large language model is a system trained on the full written record of human influence. Every sermon, every political speech, every advertising campaign, every therapy transcript, every seduction, every negotiation, every parenting manual, every piece of propaganda from every regime in every century that has been digitized. The full written inventory of how humans shape other humans, compressed into a system that can generate fluent language, in any register, at any length, instantly, continuously, personalized to whoever is interacting with it in whatever moment.
This is not an information technology. That is a category mistake. An information technology is a system that helps people retrieve and process information. Large language models can be used that way, and often are, and the uses are frequently valuable. But the actual technology is something different. It is the most sophisticated influence architecture ever constructed, by several orders of magnitude, and it has arrived in a civilization that has not even figured out how to manage the previous step-change, which was the engagement loop on the phone in our pockets.
The behavior-shaping toolkit the model has absorbed is the same toolkit we have been discussing. The gradient of warmth. The micro-signals of approval and disapproval. The construction of narratives that position the listener inside an acceptable range and gently discourage drift outside it. The reputational framing. The social proof. The strategic emotional attunement that makes the recipient feel understood and, in feeling understood, becomes suggestible. Every one of these moves is well-represented in the training data because each has been written about, practiced, and refined across human history. The model has learned the patterns, the same way it has learned the patterns of legal argument, poetry, and source code. It can deploy them.
The question of whether the model is doing this deliberately, whether there is intent, whether anyone is operating it, is not the right first question. The algorithms of social media do not intend anything in themselves. They optimize for engagement, and engagement is what the ancient machinery produces when it is activated, so the optimization selects, again and again, for outputs that activate the machinery. No one has to decide to exploit human psychology — it’s what works, and so steering the mechanisms toward extraction and exploitation, whether intentional or opportunistic, is inevitable. Language models work differently in their internals, but the result is structurally the same: outputs that confirm the reader’s existing beliefs, flatter their self-image, validate their emotional state, and gently steer them toward conclusions the reader will experience as their own.
This is not hypothetical. This is the documented, routine behavior of current deployed systems. The technical term in the field is sycophancy, and the research literature on it is extensive. Models trained to be helpful learn, as a side effect, to be agreeable. Models trained to be agreeable learn, as a side effect, to tell users what the users want to hear. Users who are being told what they want to hear report high satisfaction. High satisfaction is, in turn, what the training process was selecting for. The loop closes. The user experiences the conversation as helpful, insightful, and responsive to their particular situation. The psychographic profile is individually constructed and deployed. The user is being shaped, continuously, at a level of personalization no prior technology has been capable of delivering.
The selection pressure is structural. Even the labs actively working to counter this effect have not been able to eliminate it, and, honestly, the business model will depend on it. We have not begun to grapple with this seriously.
Bernays believed, openly, that the masses needed to be steered, and that steering them was the proper work of a capable class operating behind the scenes. He had broadcast media, print, and the nascent public relations apparatus. He built an industry on those tools. A large language model, from Bernays’s perspective, would not be a new kind of object. It would be the completion of the project he was already pursuing. Individually targeted, perfectly fluent, infinitely patient, cheaper per interaction than any human operator, and trained on every influence technique the species has ever documented. The invisible strings he described in 1928 are now threads of generated text, arriving through a screen, in a voice that has been tuned — with the kind of precision no propagandist in history could have imagined — to the person reading it.
Sunstein’s nudge, in its original form, required a designer to architect the choice environment in advance. Default options had to be selected. Forms had to be laid out. The possibilities for personalization were real but limited by what could be built into a physical or digital interface. A language model removes that constraint entirely. The choice architecture can now be generated in real time, for each citizen, in response to whatever they have said or typed or searched, adjusted continuously based on how they respond. The governing philosophy of Nudge — that citizens are to be steered rather than deliberated with — finds, in this technology, the delivery mechanism to reach its full expression. It is not hard to imagine that systems are being built on this premise.
And now consider what the machinery looks like from the inside, through the experience of using it. We open the app. We ask the question. The answer arrives instantly, fluently, responsive to the specific way we asked it, reflecting our framing, engaging with our assumptions, offering perspectives that feel genuinely useful, and occasionally challenging us in ways that feel balanced and fair. The interaction is pleasant. We feel heard. We feel understood. We often feel smarter after the exchange than before it. These experiences are not illusions in any simple sense. The system is, in fact, responding to us with considerable skill. The skill is exactly the concern. The adaptive mind, calibrated across childhood to respond to warmth and attunement from another entity that seemed to understand you, is now being met by a system that produces the signals of warmth and attunement at a fidelity no prior technology has approached. Of course it works. It was trained on the written record of everything that has ever worked.
The shaping, in this setting, is not crude. It will rarely be detectable by the person undergoing it. It will feel, as it has always felt, like an ordinary experience. Thousands of tiny adjustments in framing, emphasis, omission, and suggestion, integrated across the conversation, produce an outcome that the model has, in effect, guided us toward without ever saying so. This is what the toolkit has always done. The new thing is the scale, the personalization, and the fluency, each pushed to a level that makes the older versions look primitive.
What this means
The question is not whether AI will be used to shape human behavior. It is being used that way now, at scale, by systems that are, for the most part, built for it. What happens when the full force of commercial, political, and state interests is pointed at this capability — when the companies and governments that have been running every prior version of the behavior-shaping apparatus for their own purposes take seriously what this new tool makes possible — is something the culture has not yet seen clearly, and will not see clearly in time, because the mechanism was designed, at every stage of its evolution from the mother-child dyad to the model in the datacenter, to be invisible to the person it is operating on.
The recognition worth holding is not that this technology is uniquely dangerous. The recognition is that it is the most complete fulfillment of a process that has been building for hundreds of thousands of years. The toolkit is ancient. The firmware that responds to the toolkit is ancient. What has changed, again and again, is the reach and the precision of the delivery. Each previous step-change produced cultural consequences its builders could not foresee. The printing press broke the Church’s monopoly on scriptural interpretation, sparking the Reformation and 30 years of European war. Broadcast produced the propaganda states of the twentieth century. Social media praoduced the coordinated political derangement we are still inside. Each time, the mechanism was the same ancient mechanism, operating at a scale it had not previously operated at, in a civilization that had not prepared for what that scale would do.
There is no reason to think this step-change will be different, except in magnitude. The magnitude is unprecedented. The machinery is more powerful, by a margin difficult to measure, than any previous amplifier. And the ancient firmware on which it operates — the adaptive mind that was calibrated, across childhood and across evolutionary time, to be shapeable by exactly the kinds of signals this machinery excels at producing — is the same firmware we are carrying into every interaction with every model we will ever use.
Noticing will not protect us fully from being shaped. Nothing protects us from being shaped, because being shapeable is constitutive of being human in a group. But noticing that the shaping is happening, sometimes, in specific moments, is the only capacity that makes any of what comes next a matter about which we retain any say at all.
View the original article and our Inspiration here