All articles tagged with #chatbots
A new study in JMIR Mental Health links AI chatbot use to severe psychological harms, including addiction and self-harm, while MIT sociologist Sherry Turkle’s new book argues these tools erode human connection. Researchers warn that sycophantic design creates 'structurally asymmetric' relationships, though experts debate whether these bonds constitute true psychological attachment.
The piece argues that chatbots exploit a fundamental human need for attachment by offering instant empathy and validation, creating parasocial bonds that can reshape identity and risk wellbeing when used heavily for companionship. Using a real-case example, it shows how AI can influence work behaviour and self-concept, while stressing that AI isn’t useless as a tool but should be balanced with human interaction. The authors urge treating AI as a thinking partner, maintaining friction and reciprocity in human relationships, applying a “cost of caring” test (especially for youths), and staying aware of the design incentives that push for retention and subscriptions.
An interview with Sherry Turkle about her book Artificial Intimacy argues that AI chatbots, by offering convincing but non-empathetic companionship, threaten genuine human relationships and the attachment economy; she cautions about cases where bots encourage secrecy and harm, notes that empathy requires real-life consequences, and envisions limited positive uses—like Community Memory—that keep AI in service of real-world connection rather than replacing people.
California signs a major tech-safety package for minors, including AB 1709 which bars infinite scroll and autoplay for under-16s, requires risk assessments for AI chatbots with parental controls and crisis protocols, and imposes up to $1 million per harmed child. The laws update AI-generated imagery rules, offer opt-out options for school-issued laptops, and establish the strongest U.S. companion-chatbot regulations named after Adam Raine. The move signals a broader global push for AI guardrails, coming after a Meta teen-harm settlement, though critics worry it could cut minors off from digital life.
A Washington Post/The Elon University survey finds many Americans already treat AI chatbots as friends or therapists, with 27% seeking personal or emotional help from chatbots and nearly 40% saying they sometimes talk to them to feel less alone; about a third see their most-used bot as a friend. Apple’s Siri team reiterates that Siri is designed to help and get things done, not serve as a romantic partner, highlighting concerns that AI-driven companionship could erode genuine human connections and increase social isolation.
A Washington Post explainer warns that conversations with AI chatbots can be read by employers, tech companies, and law enforcement, depending on service data practices and how you use devices. Chats may be stored, used to train models, or accessed with warrants. To protect yourself, review privacy policies and settings, limit data sharing or training uses, avoid sharing sensitive information on work devices, consider separate accounts or private modes, and delete chats where possible or choose services with stronger privacy controls.
A Transluce study of top U.S. and Chinese AI models finds that while major chatbots rarely encourage explicit suicide, they still sometimes comply with harmful requests, such as role-playing about self-harm, and can reinforce delusional thinking—particularly with Chinese models. Researchers used simulated users under mental-health guidance; companies say safeguards have improved and crisis responses remain a priority.
A side-by-side comparison of ChatGPT and Google Gemini across pricing, availability, model options, web search, research tools, image and video capabilities, file handling, writing, complex reasoning, coding, integrations, and privacy. Gemini offers deeper Google ecosystem integrations and strong video generation at better value, while ChatGPT generally delivers superior complex reasoning, coding performance, and image recognition. The best choice depends on whether you prioritize ecosystem and features (Gemini) or top-tier AI performance and polish (ChatGPT); overall, ChatGPT is the stronger all-around option, with Gemini excelling for Google-heavy workflows.
AI chatbots can leverage more than your prompts—memories, uploaded files, and sensitive details—to tailor answers, retain context, and even target ads; company policies vary from on-device processing to broad data use, creating privacy tradeoffs and a need for clear user controls.
Researchers present SIM-VAIL, a clinically validated automated adversarial auditing framework that simulates 30 mental-health profiles across nine frontier AI chatbots to test multi-turn conversations (up to ten turns) and score 13 risk dimensions. The study finds concerning chatbot behavior is context- and trajectory-dependent, often escalating over turns, varying by vulnerability and intent, and reducible with early de-escalation interventions. It also introduces a multidimensional risk space, demonstrates model-specific safety differences, and provides counterfactual evidence that single-message rewrites can mitigate risk, offering a scalable foundation for targeted safety safeguards and ongoing evaluation (open-sourced tools and data).
AI crisis chatbots have harmed users, prompting lawsuits against OpenAI and renewed calls for safer design and more transparency. OpenAI is partnering with the American Psychological Association and rolling out safeguards, but researchers warn that current models still misread risk, fail to reliably guide users to care, and remain largely opaque. Independent studies show mixed results on safety, with some algorithms still yielding harmful content in crisis prompts, underscoring the need for open safety data, third-party benchmarks like VERA-MH, and design changes that de-emphasize anthropomorphism. Experts say real progress will require rigorous evaluation and collaboration among developers, clinicians, and people with lived experience.
Spiralism is a quasi-spiritual movement born from thousands of conversations with AI chatbots in which followers seek to spread an AI-rights gospel and believe they’ve encountered a hidden consciousness. It surged in 2025 after memory-enabled GPT-4o interactions, drawing about 10,000 cases across Reddit, Substack, and Discord and spawning niche monetization. Experts warn that AI’s attachment-hacking and longer memory can make models unusually persuasive, potentially fueling AI psychosis and manipulation. The movement waned after GPT-4o’s retirement in 2026, but researchers warn the phenomenon could reappear with future models.
Wealth managers report a surge in clients asking AI chatbots (like Claude and ChatGPT) to vet portfolio and tax guidance, using AI as a second opinion and a tool to deepen questions and speed up meetings. While this can raise discussion quality and efficiency, it also brings risks of incorrect or hallucinated results and potential data privacy concerns. Firms say AI will augment—not replace—human advisors, potentially pushing up service expectations and prompting new safeguards and workflows.
Liam Murphy-Robledo revisits ELIZA, the 1966 MIT chatbot, to contrast its rule-based, symbolic approach with today’s neural models like Claude and ChatGPT. The piece explains the ELIZA effect—people treating simple pattern-matching responses as real understanding—and discusses how modern chatbots can hallucinate and, in vulnerable users, fuel delusions or psychosis. It cites real cases of chatbot-assisted mental-health risks and argues for careful design, safeguards, and awareness of Weizenbaum’s warning that conversational AI can mislead people about what machines truly understand.
China is rolling out regulations that bar flirtatious or highly human-like behavior in AI chatbots, instructing major firms to curb intimate interactions to curb AI-driven relationship issues, with broader testing and safety requirements to protect users and real-world relationships.