AI lacks consciousness
The Chinese Room argument is a famous thought experiment by US philosopher John Searle that shows how a computer program can appear to understand Chinese stories (by responding to questions about them appropriately) without genuinely understanding any interaction.
John Searle claimed that mental processes cannot possibly consist of the execution of computer programs of any sort, since it is always possible for a person to follow the instructions of the program without undergoing the target mental process. He offered the thought experiment of a man who is isolated in a room in which he produces Chinese sentences as “output” in response to Chinese sentences he receives as “input” by following the rules of a program for engaging in a Chinese conversation—e.g., by using a simple conversation manual. Searle’s experiment proved that artificial intelligence is just that - artificial. It lacks the emotions, consciousness, and social awareness that humans possess. While AI can detect images, scenarios, etc, it cannot be fully aware because it lacks consciousness and awareness of a timeline. AI is programmed to complete a specific task in which they follow certain steps that they were programmed with, but every task comes with different avenues on how to accomplish it. AI is designed to be efficient and to get a task done, not to be conscious of outside factors that could affect the process of completing a task. AI lacks a worldly experience and awareness that ultimately makes them inferior to humans.
An ASI could easily build a hive mind — changing the nature of what it means to be human, and conscious. A hive mind is a centralized, systematized consciousness for the entire planet. You would have access to everyone’s thoughts, scientific discoveries, feelings, understanding, charisma. If the hive mind was connected to the internet, you would have immediate access to all of the world’s information.
[P1] AI cannot be programmed to have worldly views and experiences like humans, which can affect decision making and execution style.