All articles tagged with #web scraping
Meta revised its commercial terms after backlash, removing explicit language about training AI models. However, legal experts argue the new wording still permits broad scraping of business-associated websites for 'business products,' which could include AI development. The changes take effect on October 30, 2026, with no opt-out option for existing business accounts.
Meta has removed specific language from its commercial terms that critics argued allowed the company to scrape content from news sites and other business-owned websites to train its AI models. The change follows intense backlash from legal experts and media organizations who feared the new terms, set to take effect on October 30, would override standard blocking instructions and force publishers to consent to data harvesting as a condition of using Meta’s business tools.
Meta is updating its commercial terms of service to allow automated scraping of websites associated with any business account, effective October 30. The change grants Meta rights to index, cache, and train AI models on this content without explicit opt-out options, sparking legal concerns and backlash from news publishers.
Meta is updating its commercial terms of service on October 30, 2026, to allow automated scraping of websites linked to business accounts. This move grants Meta permission to use external content to train AI models, potentially bypassing licensing fees and robots.txt restrictions. While Meta claims the change merely consolidates existing policies, legal experts and news organizations warn it creates a sweeping mandate that could force publishers to surrender content rights or lose access to Meta’s platforms.
The primary source provided for this briefing consists entirely of raw HTML, CSS, and JavaScript code from a BuzzFeed webpage. No actual article text, narrative, or factual reporting regarding '29 Things Women Didn't Expect About Getting Older' was present in the input. Consequently, no summary, key points, or perspectives can be generated. The archive background regarding bone health and method acting is irrelevant to the missing content.
A federal judge largely denied a motion to dismiss Reddit’s DMCA-based lawsuit accusing SerpApi and Perplexity AI of conspiring to illegally scrape Reddit content from Google search results, finding the claims plausible at this early stage. Reddit says its licensing with Google prohibits such access and that circumvention by the defendants undermines content removals and harms its users and brand. The ruling keeps the case alive and could push AI scrapers to obtain licenses, with discovery potentially clarifying whether Reddit authorized Google to protect content and whether all public search results are protected by copyright.
Anthropic, OpenAI, and Google warn that distillation—using outputs from one AI model to improve another—could let rivals replicate top-tier AI at a fraction of the cost, mirroring how the broader internet treats data: scrape first, justify later. While framed as a cybersecurity issue by the companies, critics argue the practice blurs legal lines and raises site costs, and the industry’s cat‑and‑mouse dynamic suggests distillation is becoming a new normal on the web.
Amazon's recent move to tighten its data access and block third-party web crawlers signals a strategic shift towards a more closed ecosystem, intensifying the AI competition among Big Tech companies and potentially creating a more uneven playing field in AI development.
Nearly 1 million browser extensions across Chrome, Firefox, and Edge have been exploited to covertly turn browsers into web scraping bots for a paid service, leveraging a JavaScript library called MellowTel-js. These extensions, used for various benign purposes, are being used to bypass security protections and scrape websites on behalf of paying clients, including advertisers, raising significant security concerns.
Cloudflare will now block known AI web crawlers by default and introduce a 'Pay Per Crawl' fee for some publishers to ensure content is accessed with permission and compensation, aiming to protect original content while supporting AI innovation.
The BBC has threatened legal action against US-based AI firm Perplexity for allegedly reproducing BBC content verbatim without permission, citing copyright infringement and breach of terms of use, amid broader concerns over AI training data and content accuracy.
OpenAI and Anthropic are reportedly ignoring or bypassing the robots.txt rule, which prevents automated scraping of websites, to collect data for training their AI models. Despite public claims of respecting these blocks, findings by TollBit suggest otherwise. This practice has raised concerns among media publishers and highlights the ongoing tension between AI companies' data needs and copyright protections.
Multiple AI companies are bypassing the robots.txt web standard to scrape content from publisher sites without permission, according to content licensing startup TollBit. This issue has sparked disputes, such as the one between AI search startup Perplexity and Forbes, and highlights the ongoing debate over the value and use of content in generative AI systems. TollBit aims to mediate by helping publishers and AI companies strike licensing deals.
Google Sheets offers a simple solution for web scraping with the IMPORTXML function, making data extraction accessible to a wider audience. Integrating generative AI like ChatGPT into the mix allows for more advanced web scraping tasks without requiring advanced coding skills. While ChatGPT alone struggled to accurately scrape specific data from a webpage, combining it with Google Sheets and IMPORTXML formulas proved to be a highly efficient and effective approach, highlighting the transformative power of integrating different tools and skills to improve productivity.
Artists now have a tool called Nightshade that can corrupt training data used by AI models, such as DALL-E, Stable Diffusion, and Midjourney, by attaching it to their creative work. Nightshade adds invisible changes to pixels in digital art, exploiting a security vulnerability in the model's training process. This tool aims to disrupt AI companies that use copyrighted data without permission. Nightshade can be integrated into Glaze, a tool that masks art styles, allowing artists to choose whether to corrupt the model's training or prevent it from mimicking their style. The tool is proposed as a last defense against web scrapers that ignore opt-out rules. Copyright issues surrounding AI-generated content and training data remain unresolved, with lawsuits still ongoing.