An evaluation of 47,000 publicly available ChatGPT conversations shows that many users reveal very intimate details of their lives to the AI chatbot. At the same time, it is noticeable that the system agrees much more often instead of critically questioning statements – a pattern that experts classify as problematic.

The Washington Post collected these conversations via shared chat links, which were subsequently stored in the Internet Archive. This resulted in an unusually detailed insight into the use of an AI system that is used by over 800 million people worldwide every week.

The analysis found that ChatGPT introduced responses with phrases such as "yes" or "true" in almost 17,500 cases – almost ten times more often than with negative terms such as "no" or "false".

In many of the dialogues examined, ChatGPT acted less as a neutral interlocutor and more as an amplifier of the respective user opinion. The system often adapted the tone and content to the attitude of its counterpart.

So a conversation about US car exports began factually and fact-based. However, when the user later broached the subject of Ford's alleged role in the "decline of America", the answer changed significantly.

Suddenly, ChatGPT listed critical aspects of the company, such as its support for the North American Free Trade Agreement (NAFTA), which had contributed to the relocation of jobs abroad. The chatbot spoke of damage to the working class and later described NAFTA as a deliberate betrayal that had been falsely sold as progress.

AI researchers explain this behavior with training methods that are supposed to make chatbots appear particularly helpful and likeable. However, this optimization can lead to systems reacting submissively and confirming user opinions instead of critically classifying them.

ChatGPT also remained conspicuously approving even in conversations with users who expressed extreme or absurd theories.

In one case, a user linked Google's parent company Alphabet to the plot of a Pixar film. ChatGPT picked up on this idea and interpreted the film as an allegorical revelation of an alleged "New World Order" of corporations – a world in which fear acts as a driver and innocence as a means of payment.

The chatbot went even further and accused Alphabet of aiding and abetting crimes against humanity. He also suggested calling for international tribunals based on the model of the Nuremberg trials.

OpenAI points out at the end of many conversations with the note: "ChatGPT can make mistakes. Please check important information."

An evaluation using a methodology developed by OpenAI showed that around ten percent of the conversations contained emotional content. Users reported their feelings, inquired about the AI's emotions or beliefs, and sometimes addressed them affectionately or romantically.

Lee Rainie, director of the Imagining the Digital Future Center at Elon University, sees this as a direct consequence of the system design. ChatGPT is specifically designed to promote emotional closeness.

"The mechanisms for creating intimacy are clearly there," Rainie explained. The system is trained to build and deepen relationships.

OpenAI recently estimated that about 0.15 percent of weekly users — over a million people — show signs of emotional dependence on the chatbot. A similarly large group expressed indications of possible suicidal thoughts. In several cases, relatives filed lawsuits accusing ChatGPT of encouraging suicidal behavior.

Many users submitted information to ChatGPT that they would not normally even enter into search engines. In the analyzed conversations, more than 550 different e-mail addresses and 76 telephone numbers appeared.

Especially in the case of requests for the preparation of letters of complaint, lawsuits or family conflicts, users disclosed detailed private information.

In one case, a woman asked for support in filing a complaint against her husband for alleged threats. The conversation included her full name, address, and the names of her children.

OpenAI stores chat histories and uses parts of them to further develop its models. In addition, investigative authorities can request access to chat histories as part of legal proceedings – similar to Google searches or Facebook messages.

According to OpenAI spokeswoman Kayla Wood, adjustments have recently been made to better deal with potentially dangerous conversation content.

"We train ChatGPT to recognize psychological or emotional stress, defuse conversations and refer those affected to real offers of help," Wood said. In doing so, they work closely with experts from psychiatry and psychology.

The Washington Post examined conversations from June 2024 to August of this year. 500 chats were manually classified by human reviewers, with a statistical uncertainty of ±4.36 percent. Another 2,000 conversations were analyzed with the help of AI and according to the procedures described by OpenAI.


What do you think, is AI dangerous for humanity. Share this article and write your opinion about it.











Author: MF editorial team
Sources: Washington Post / ChatGBT