All articles tagged with #reproducibility
Stanford researchers introduce Paper2Agent, a framework that turns scientific papers and their data, code, and workflows into autonomous AI agents capable of discussing findings, reproducing analyses, applying methods to new data, and collaborating with other agents to accelerate scientific discovery—while emphasizing validation and caution about AI hallucinations.
Nature reports on Paper2Agent, a system that ingests a paper’s text, code, and data, hosts it on an MCP server, builds a paper-specific AI agent, and lets researchers interact in natural language; the agent can apply the paper’s methods to new data and even collaborate with agents from other disciplines. In tests on the AlphaGenome paper (DNA sequence predictors), the agent was created in about 45 minutes at a cost of ~$14 and achieved near-perfect accuracy on genetics questions, outperforming other biomedical AI agents and enabling reanalysis of conclusions without new experiments.
A new framework called Paper2Agent converts scientific papers into autonomous AI agents by packaging a paper’s manuscript, data, code and workflows into a Model Context Protocol (MCP) server and linking it to chat agents; case studies with AlphaGenome, Scanpy, and TISSUE show agents reproduce results and handle novel, user-asked analyses, lowering barriers to adoption and enabling collaborative AI co-scientists.
This Perspective argues that deciphering microbiome function requires coordinated, benchmarked, FAIR multi-omics workflows spanning metagenomics, metatranscriptomics, metaproteomics, and metabolomics. It reviews each layer’s strengths and limitations, highlights cross-omics integration challenges, and proposes a practical, tiered roadmap—foundational study design, integration and benchmarking, and ecosystem infrastructure—driven by community standards and open benchmarks to enable reproducible, interpretable, and AI-ready insights across diverse environments.
Science sleuth Reese Richardson has flagged more than 18,000 questionable validation images across 15 antibody vendors, expanding May’s Thermo Fisher findings. Automated image–manipulation checks and manual review reveal repeated backgrounds and other edits on validation images, suggesting many catalog entries may not reflect tested performance. Four companies account for most flags; Thermo Fisher and LSBio have issued statements, while others did not comment. While not proving defects, the results raise serious questions about the reliability of commercial antibodies and highlight ongoing reproducibility and cost concerns in biomedical research.
AI agents are increasingly auditing science literature, exposing decades-old errors in a long-trusted chemistry reference database and in conference papers. While AI can speed up error detection and reproducibility checks, human oversight remains essential due to occasional mistakes and misreadings.
Nature reports that PubPeer is launching a project to post replication studies on its platform and link them to the original papers, in an effort to make replication efforts more visible and speed scientific self-correction. The plan involves about 2,400 replication studies added in batches of ~100 via the FORRT Library of Reproduction and Replication Attempts (FLoRA), highlighting both successful and failed replications. Authors of the papers are being notified, with mixed expectations about PubPeer comments, but supporters argue that publicly confirming a study’s validity can benefit the research community by reducing wasted effort and improving credibility.
The article shows how researchers use everyday kitchen items and simple gear to make field science more robust, reproducible, and accessible: a soup ladle on a pole and a strainer to collect and clean brine samples; a jewellery chain to estimate soil roughness; and kite-based surveys as durable, low-cost alternatives to drones. It emphasizes improvisation in remote work, contrasts high-tech and low-tech methods, and highlights global collaborations (like CrustNet) built on shared, widely available protocols to democratize scientific data collection across diverse sites.
A consensus-based GUIDE-LLM checklist (14 items) has been developed to boost transparency, reproducibility, and ethical accountability in research using large language models in behavioral and social science. Created via a preregistered two-round Delphi with international experts, it covers when and how LLMs are used, model details and prompts, data inputs and privacy, validation, reproducibility, and disclosure of competing interests. While broadly applicable, the checklist allows context-specific flexibility and is maintained as a living document, with optional items and guidance to share code and interactions (redacting sensitive data) to enable verification and adaptation by others.
Large, cross-lab projects in infant- and animal-cognition are being used to tackle the psychology reproducibility crisis, with initiatives like ManyBabies, ManyDogs, and ManyBirds increasing statistical power and diversity, though results have been mixed and sometimes contradict earlier findings.
Eighteen teams analyzed the same Neuropixels dataset and largely disagreed on ripple density across brain areas, despite using defensible methods. The divergence arose from differences in how concepts were defined, which algorithms were used, and the parameters chosen, revealing substantial analytical variability. The effort spurs the CON²PHYS project to quantify conceptual disagreement and push for transparency, reference pipelines, and reporting standards to ensure conclusions are robust to analytical choices.
An NIST redo of the 2007 BIPM measurement of the gravitational constant G, using a blinded-envelope approach to avoid bias, yields a result close to the French value but with a 0.0235% discrepancy after adjustments; Schlamminger also identifies a newly observed spurious torque driven by temperature gradients and residual gas in the vacuum, suggesting unaccounted biases in the uncertainty budget and underscoring the ongoing challenge of precisely measuring G and the importance of reproducibility.
An opinion piece cautions that rapid, uncritical adoption of AI and large language models in science is boosting output while narrowing inquiry, risking lower-quality results and erosion of tacit training for early-career researchers. It calls for guardrails to preserve hands-on apprenticeship, ensure responsible oversight of AI-assisted workflows, and use metrics that reflect true scientific understanding rather than sheer productivity.
A genome-wide check of 611 samples from 341 mouse strains in the NIH/MMRRC network found that 47% did not match their reported identities, exposing widespread mislabeling and genetic drift that could undermine the reproducibility of studies relying on these models.
A Europe-backed NanoBubbles project is funding nanoscientists to replicate a 2012 study that carbon quantum dots can sense copper ions inside living cells, the first large-scale replication effort in the physical sciences aimed at the reproducibility crisis; initial attempts failed to reproduce the reported fluorescence change, illustrating how small impurities, incomplete protocols, and cross-lab variation can affect results, as the ERC-backed effort seeks self-correction in science.