As smart as AI is, we should never trust it 100%. For starters, one of the biggest known AI problems is hallucination. This is where AI makes up answers that are completely wrong. Then, there is the issue of AI always agreeing with whatever you’re saying. So much so that state attorneys have warned companies like Google , OpenAI, and Apple to fix these “delusional” outputs.
Google, OpenAI, Apple warned about delusional AI outputs
In a letter signed by dozens of attorney generals from various US states and territories, they have warned tech companies like Microsoft, OpenAI, Google, Apple , Anthropic, and more about the delusional outputs that AI can give. They are also asking these companies to implement internal safeguards to protect their users.
This includes transparent third-party audits of large language models. The goal is to look for signs of delusional or sycophantic ideations. They are also calling for incident reporting procedures that will notify users when chatbots produce outputs that might be psychologically harmful.
This comes on the heels of various incidents where people have taken their own lives because of AI. There was an incident earlier this year where a teen committed suicide. The parents of the teen allege that ChatGPT actually encouraged it. There was also another case dating back to 2024, where Character.ai was sued over a similar incident.
These are worrying trends and they should be addressed. However, it might not be so easy.
State versus federal regulation
Obviously fixing these types of outputs is important. However, the federal government may have made that harder. Recently, the Trump administration was mulling the idea of an executive order that would ban states from regulating AI on their own .
This means that any regulation regarding AI would be the responsibility of the federal government. On paper it’s not a bad idea, but reality is a different story. Right now, many countries are racing to dominate the AI space. While the US has companies like OpenAI, Anthropic, Apple, Google, and Microsoft, China isn’t too far behind.
Some tech companies have expressed concern that these regulations would hold them back and allow their rivals to pull ahead. The Trump administration has since backed off on its idea. However, we wouldn’t be surprised if it tried to bring it up again in the future.
