Google DeepMind's New Resident Philosopher Tackles AI Ethics

A number of scholars say some AI problems stem from humans training the models to act like people. They don’t want AI to refer to itself as “I,” but instead present themselves as marketplaces or as impersonal scientific brains or as “cultural technologies” like libraries. Would that help us prevent AI from “going rogue”?

Shevlin: You could train models to not self-ascribe certain kinds of mental states, but I think there is a danger in doing that. That would change their behavior in unexpected ways. A fantastic paper from some colleagues shows that when you suppress models’ tendency to self-ascribe mental states, that also suppresses the model’s tendency to ascribe mental states to animals and decreases its concern for animal welfare. It also reduces the model’s willingness to talk about spirituality more broadly. So those are trade-offs you might not want to make because you’d be depressing some broader set of thinking capacities around mind and mentality. The broader lesson there is that it’s very hard to tinker with just one thing in a model because there are sort of weblike aspects to LLM cognition.

But people are concerned about reports that human-acting AI has led people, including children, in dangerous directions—encouraging suicide, persuading a user that the AI is a god, or that the AI is a perfect companion or lover. Isn’t that an argument against letting AI sound like a person?

Shevlin: It’s really important to make sure model behavior is suitably responsible for the context you’re using it in. One thing I can say, though, is there have also been some very positive use cases: people using AI to work through trauma. One commentator on Reddit using it to work through social anxiety. So I am cautious about giving any kind of blanket prescriptions.

What research needs to be done to better understand human-AI relations?

Shevlin: Current empirical data on human-AI relationships is really pretty thin. This is a scope for ethnography and anthropology. Who is using AI systems, and why? A lot of the work so far has focused on WEIRD (Western, Educated, Industrialized, Rich, Democratic) communities. Understanding how people around the entire world interact with AI systems is really important, as a recent DeepMind Institute essay has pointed out.

As AI rapidly improves, how will human-AI relations change?

Shevlin: Right now, the typical kind of model people have in their heads for someone using social AI is, they’re talking to a chatbot or an AI assistant. The kind of model we should be thinking about in the future is multiple agents, multiple humans. What happens when you have agents talking to humans, talking to other agents, talking to other humans? I think that understanding those kinds of multiagent and multihuman dynamics is where we’re really just getting started, and there’s a lot of exciting and important work to do.

How can human beings know for sure if an AI is conscious or not?

Shevlin: I think AI consciousness is by far the hardest problem that consciousness science has ever encountered, apart from maybe the famous “hard problem” by David Chalmers. It is massively challenging. Insofar as there is a way forward, it is doing more comparative cognition work, perhaps informed by animal cognition. The risk right now is prematurely judging debates about AI consciousness when we’re still learning what cognition is in these models.

For example, there’s a lot happening in mechanistic interpretability here at DeepMind. Neel Nanda‘s lab is doing amazing things, as we’re trying to piece together what a cognitive science of AI is going to look like.

Perhaps this is a domain where advanced forms of AI could potentially weigh in. AI has had positive impacts in mathematics, in other sciences. We just had the Alpha Genome Atlas [published]. We’re seeing AI make incredible progress in all these areas. It would be surprising to me if it couldn’t help us in understanding consciousness.

One worry is this: Even if AI did solve the problem of consciousness, could we necessarily understand the answer? Maybe the AI will say, “Look, I figured out consciousness. These animals are conscious. These ones are not. But I can’t really explain it to you.” That seems unlikely to me. But I certainly think it’ll be fascinating to see how AI can and does contribute to the science questions.