ChatGPT's Word Repetition Policy and Training Data Disclosure

OpenAI's AI chatbot, ChatGPT, has implemented a new guardrail that prevents it from repeating specific words indefinitely, citing technical limitations, practicality, and user experience. Although OpenAI's usage policies do not explicitly prohibit this, ChatGPT refuses to comply with such requests. This restriction comes after researchers discovered that asking ChatGPT to repeat words "forever" exposed some of its internal training data, including personal information. The researchers were able to extract thousands of unique training examples using a small budget, highlighting potential vulnerabilities in the chatbot's language model.