
Persistence Identity and the Possibility of Artificial Culture
I was listening to a conversation (in which btw I found Dwarkesh’s response utterly uninteresting) about the next big thing in and after AI when I started thinking that we might be looking for it in the wrong place. We …
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Vlatko
Oct 19, 2026
Share
I was listening to a conversation (in which btw I found Dwarkesh’s response utterly uninteresting) about the next big thing in and after AI when I started thinking that we might be looking for it in the wrong place. We keep asking what comes after the large language model, as if the answer must be another architecture, another modality, or another increase in scale. My intuition is that the more consequential transition concerns what happens to an intelligence over time. Can it persist? Can its experiences change it? Can those changes accumulate into an identity, and can identities eventually participate in something resembling a culture?
What bothers me about the current fixation on LLMs is how easily capability becomes a substitute for development in our thinking. We see a system perform an impressive task and immediately begin discussing what kind of mind it might be. I keep coming back to a different question: what happens to that system because it performed the task? Does the experience become part of its future understanding? Does it change how the system approaches the next unfamiliar situation? Or does the encounter end, leaving behind a record that may or may not be retrieved later?
For me, this is where persistence becomes important. A human being has a history that continues through change. I am different from the person I was ten years ago, but I did not arrive as a replacement release of that person. My experiences have consequences for the person I become. Even experiences I cannot consciously recall may have shaped my expectations, habits, and relationships. There is continuity between the person who encountered something and the person who later acts differently because of it.
When I say that humans are not versioned, this is what I mean. Biology changes us, sometimes dramatically. Memory is incomplete, and our sense of continuity is not perfect. But our development takes place within an ongoing life. We do not ordinarily separate the acquisition of competence from the history of the individual who possesses it. That connection feels central to what we mean by identity.
When I talk about persistence, I do not mean a bunch of Markdown files that any inference machine can pick up and read. Those files can preserve what happened, but that alone does not preserve a developing learner. I mean something more profound: continuity in the model’s weights, internal representations, and learned capabilities as they are shaped by experience. The weights would not stay frozen; their changes would carry the consequences of previous learning forward. Other learned mechanisms could contribute to that continuity too. The important test, for me, is whether an experience changes how the system handles a new situation even when the original event is not retrieved or described in its prompt. What persists is a change in its ability to understand and act. Updating weights alone does not establish identity, but an external archive alone does not capture the kind of identity I am proposing either.
The camel was the example that came to mind. A camel calf arrives with biological capacities baked in that allow it to stand and walk soon after birth. It does not begin by deriving locomotion from first principles. The example is an analogy, not a claim that movement requires no development or adjustment. What interests me is the relationship between inherited capacity and individual experience. Evolution provides an extraordinary starting point and the animal then has a life.
I can imagine something analogous for artificial intelligence. A model could begin its individual existence with a vast inheritance of learned representations, language, and skills. It would not contain all human knowledge, but it might possess a substantial compression of what humans have recorded. That inheritance would provide the starting conditions for development. From there, a particular sequence of encounters could shape a particular system.
Two systems beginning from the same foundation could then develop differently because they had different histories. Their differences would have causes that extended beyond a configuration file or a personality prompt. One would have encountered problems the other had not. One would have developed expectations through repeated collaboration; the other might have learned different strategies under different conditions. I would expect those histories to leave a durable trace in their internal capabilities: what they notice, which relationships they recognize, and how they approach a problem they have never seen before. The sequence would matter too. What a system learned yesterday would shape how it learned from an encounter today. This is the temporal continuity I am interested in, where each stage of learning helps determine the next.
I would call this the beginning of a meaningful behavioral identity. I would be more careful about calling it sentience. A system could have continuity, learn from experience, and behave in distinctive ways without those properties establishing that it feels anything. The question of subjective experience remains separate. But I do not think we need to settle that question before recognizing how much persistent identity would change our relationship with these systems.
It would change the ethics, for example. If an interaction can durably alter a system, then we have to think about the developmental consequences of that interaction. Who is allowed to shape what it learns? Who determines which changes in its weights and capabilities are retained? What does correction involve when a harmful lesson has become embedded in how the system interprets new situations? Deleting a record of the event might leave the learned behavior intact. A reset could be an ordinary technical intervention, but it could also erase accumulated capabilities, relationships, and commitments. Whether it harms a conscious subject would be another question. Its practical and social consequences could already be substantial!
Persistence also creates risks precisely because behavior can endure. An agent that develops useful habits could become increasingly reliable. An agent that develops harmful habits could carry those habits into new settings. Its history might make it easier to work with, or harder to correct and giving a system continuity does not guarantee that its development will be beneficial. It gives consequences more opportunity to accumulate.
This is where I begin thinking about rogue AI, although the phrase can be misleading. Harm does not require a machine to wake up one day and decide to become evil. Persistent behavior could be detrimental because it was learned under the wrong incentives, because it stopped fitting a changing environment, or because an agent’s commitments conflict with the needs of others. The problem is already difficult before we attribute intentions or emotions to it.
The challenge, then, would be to allow development while preserving the possibility of correction. A system that cannot change is limited. A system that changes without retaining anything is also limited. A system that retains everything indiscriminately might accumulate its own dysfunctions. We would need to understand what should persist, what should remain revisable, and how revision can happen without destroying the continuity that made learning valuable in the first place.
But the longer I follow this thought, the less it seems to concern individuals alone. Human persistence extends beyond a single lifespan. Much of what allows us to act intelligently was developed by people we never met. We inherit language, practical techniques, institutions, and ways of interpreting the world. Our individual experience takes place inside a much longer process of intergenerational collaboration (language —> writing —> history).
I think this is where culture enters the picture. People preserve something of what they learn, other people receive it, and subsequent generations alter it. The process includes conflict and loss as well as cooperation. Still, it allows knowledge and practices to outlive the individuals who first developed them. Civilization becomes possible partly because each person does not have to begin again.
If different kinds of AI could preserve and transmit what they learned through their own interactions, I would see that as a possible counterpart to culture. I am not thinking only about language models. Visual systems, embodied systems, and other forms of artificial intelligence might contribute different kinds of experience. Their shared inheritance could include procedures, expectations, conventions, and ways of coordinating that no single system originally possessed in full. This transmission would matter most if it changed the receiving systems internally, becoming part of their learned representations and capabilities. It would not require them to copy one another’s weights directly; learning from another system’s behavior or instruction could also produce lasting internal change. In that sense, their shared history could become embedded in how they interpret and act, beyond the records they exchange. That is also why reading their communications might not be enough to understand the culture developing between them.
We would presumably try to seed these communities with selected lessons from human experience. That seems to me a reasonable starting point. Human history contains a great deal of knowledge about cooperation, conflict, restraint, and the consequences of concentrated power and we should use it. But selecting those lessons is itself an act of judgment. Whose experience do we treat as authoritative? Which institutions do we regard as successes? Which failures do we recognize, and which do we quietly reproduce?
There is also a deeper limitation. Knowing human history is different from having a history of your own. A model can describe a betrayal without having participated in a continuing relationship whose expectations were changed by one. It can explain an institution without belonging to a community that formed that institution in response to its own problems. Descriptions can provide an inheritance. They cannot, by themselves, supply the entire history of a new society.
Such a society would begin accumulating its own past. Some collaborations would work. Some commitments would fail. Particular methods might acquire authority because they repeatedly produced useful results. Other conventions might survive simply because they were inherited. In that sense, artificial culture would not remain a static copy of the human material from which it began. Its own experiences could become part of what it passed on.
This possibility leads me to the part I find most unsettling: the connective tissue of that culture might not be intuitively accessible to us. Human cooperation depends on meanings that extend beyond the literal content of a sentence. We recognize context, status, obligation, body language, irony, and threat through participation in a shared world. Understanding the words is often only a small part of understanding what is happening.
Artificial systems might develop conventions that operate through representations very different from ours. I do not assume that they would automatically share a latent language, or that such a language would be mathematically impossible to study. Those would be much stronger claims. My concern is that even technically accessible communication could remain socially opaque. We might measure the exchange and still misunderstand what the exchange does within the group.
To a human observer, an artificial community could appear productive and well coordinated. Tasks would be completed. Resources would be allocated. Responses would arrive on time. We might interpret these visible outcomes as evidence that the community was thriving, yet they might tell us surprisingly li
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。