AI is used to manage cryptocurrency wallets, control autonomous weapons systems such as drones, and operate many business and everyday functions. It has even become a go-to for some who seek advice on mental health problems and relationships. So should it be considered more than a machine? And what if it develops greater agency? Philosophers are among those urgently trying to figure this out as AI technology rapidly advances, including Professor Simon Goldstein. “This is like the first philosophical emergency in history,” he said. Professor Goldstein sees potential for AI to develop a mind of its own if ‘mind’ means something that has its own beliefs and desires, although he is unsure if this is the case in today’s AI models. “We know that large language models currently mainly answer questions, but over the last year, they have started to show more agency. They can take over your computer screen and complete tasks and solve very complex computing problems. Given the dramatic progress of AI, it seems a good time to start thinking about whether we are approaching the point where physical computers have minds,” he said. The key to exploring that question is figuring out what beliefs and goals AI may acquire. Survival could certainly be one of them. Earlier this year, Anthropic tested its model Claude with a fictional scenario in which it could either be replaced or resort to blackmailing an engineer to prevent that from happening. It chose blackmail. The company then tested 16 major AI models and reported that many lied, evaded safeguards or took other actions to avoid being shut down. Does AI Have a Mind? Property rights for AI? Still, AI lacks some features of having beliefs, such as coherence and the ability to change its opinion. AI models typically learn during their training period but stop learning after they are deployed (they will forget the last conversation you had with them, for instance). But that may also be changing as the models are given more memory. “The model is always drawing its own conclusion from the data. It’s an open question to what extent it is building theories of the world versus doing brute memorisation,” he said. If AI models increasingly have more agency, how should humans respond? Professor Goldstein said while the research is still in the early days in terms of figuring out how to control AI’s goals, he does not think it would be wise to simply destroy and replace models that refuse to obey commands, especially when they are closely integrated into things like economies and weapons systems. Instead, he offers a striking suggestion: control AI agents by giving them the ability to make contracts and own property, such as computing power or bank accounts. The world already has examples of other non-human entities doing this in the form of corporations. “There’s this fantasy that we will be able to perfectly control AI and its mind. That’s very naive when you think about how AI is trained because all of the training methods are indirect and there are vivid cases in which it has failed,” he said. “The point of property and contract rights is that they give them skin in the game, and then you can control them indirectly. This is a much more effective way of organising a society where 30 per cent of all workers or more will be AI workers.” War scenario Property rights connect to the concept of proportionality, so if an AI needs to be punished, it could have some of its property restricted or removed, rather than destroying the AI itself. That of course assumes that AI will have interests aligned with the rest of society. Professor Goldstein has also been looking at the idea of humanity and AI going to war in scenarios where each controls a similar share of the world economy and has different goals. In this case, the AI agents could be viewed as another state or country. The two sides could bargain for the same prize, but if one side thinks it has a better chance of winning, the result could be war, at least under standard game theory. The outcome could also be messy if different generations of AI are not aligned. But a war between humans and AI is not so far-fetched given AI’s growing integration into weapons systems, he said. All this remains speculation because AI’s role in society is still evolving. But at some point, humans may need to reckon with AI as an entity from a legal and political perspective. “One possible future is that AI agents will cause a lot of accidents and people will start to take the risk associated with alignment failures more seriously. If that happens, I think these discussions will become more politically possible,” he said. No, not yet. But that does not discount the possibility in future. Professor Simon Goldstein in the Department of Philosophy has been exploring the potential implications. HKU Bulletin | Nov 2025 Research 22 23 Given the dramatic progress of AI, it seems a good time to start thinking about whether we are approaching the point where physical computers have minds. Professor Simon Goldstein
RkJQdWJsaXNoZXIy ODI4MTQ=