tldrdailynews.com logo
Tag

Inference

All articles tagged with #inference

SambaNova valued at $11B after $1B funding to accelerate on-prem AI inference
technology2 days ago

SambaNova valued at $11B after $1B funding to accelerate on-prem AI inference

SambaNova raised $1 billion in fresh financing led by General Atlantic (with Seligman Ventures, T. Rowe Price and Capital Group), boosting its valuation to $11 billion and fueling the rollout of its SN50 inference chips for on‑premise deployments. JPMorgan has adopted the company's systems for demanding enterprise AI workloads, while the startup—which earlier drew funding from Intel—continues to pursue a U.S. IPO around 2027 amid strong investor interest in AI-chip companies challenging Nvidia.

via CNBC|
#ai-chips#inference#ipo
OpenAI and Broadcom unveil Jalapeño, a data-center chip for scalable LLM inference
technology16 days ago

OpenAI and Broadcom unveil Jalapeño, a data-center chip for scalable LLM inference

OpenAI and Broadcom introduced Jalapeño, a purpose-built ASIC designed from scratch for large-language-model inference in data centers, with early testing claiming substantially better performance per watt; development took nine months and is part of a broader effort to own more of the AI stack and reduce reliance on Nvidia, with deployments planned by year-end as the silicon race heats up.

via Ars Technica|
#broadcom#inference#jalapeno
Intel bets on a cheaper AI inference GPU to shake up data-center chips this year
technology1 month ago

Intel bets on a cheaper AI inference GPU to shake up data-center chips this year

Intel plans to ship its Crescent Island AI data-centre GPU by year-end to accelerate inference, using cheaper LPDDR5 memory and air cooling to cut cost vs Nvidia/AMD. Led by Kevork Kechichian, it marks Intel’s first major AI infrastructure push under CEO Lip-Bu Tan, with limited initial shipments after an 18-month development. The chip targets cheaper memory and cooling, aims to compete on price and power, and Intel is pursuing in-house foundry plans with potential China sales under export controls as it rebuilds its AI hardware business.

via Financial Times|
#artificial-intelligence#gpus#inference
Memory Over Speed: The AI Inference Shift
technology1 month ago

Memory Over Speed: The AI Inference Shift

Ben Thompson argues that the AI compute boom is moving from GPU-dominated training to memory-centric, agentic-inference architectures; Cerebras’ wafer-scale chips offer extraordinary on-chip memory and bandwidth for fast answer inference but face cost and scalability limits, while the long-term potential lies in memory hierarchies that support autonomous agentic work, potentially reducing Nvidia’s dominance and reconfiguring compute across training, inference, and even space data centers.

via Stratechery by Ben Thompson|
#agentic-inference#ai-hardware#cerebras
Nvidia Unveils Groq-Enhanced Inference to Defend AI Chip Lead
technology3 months ago

Nvidia Unveils Groq-Enhanced Inference to Defend AI Chip Lead

At its GTC conference, Nvidia unveiled a product that pairs its chips with Groq’s acceleration tech to boost AI inference speed and cut costs, a move aimed at defending its dominant hardware position as rivals advance. The announcement follows Nvidia’s $20 billion Groq licensing deal and includes NemoClaw to help software companies deploy AI agents, all while supply-chain and manufacturing constraints shape growth prospects for its Rubin and Blackwell line.

via The New York Times|
#chips#groq#gtc
Nvidia-OpenAI $100B plan fizzles into non-binding talks
technology5 months ago

Nvidia-OpenAI $100B plan fizzles into non-binding talks

A September 2025 letter of intent for Nvidia to invest up to $100 billion in OpenAI’s AI infrastructure has not materialized; Nvidia’s Jensen Huang says the figure was never a commitment, and Reuters reports OpenAI has been seeking alternatives and citing Nvidia chip speed concerns for inference. OpenAI has since struck deals with Cerebras, Groq, AMD, and Broadcom to diversify compute, while Nvidia emphasizes a large future investment but not at that scale. The news triggered a stock dip for Nvidia and highlighted questions about timing and strategic fit.

via Ars Technica|
#ai-chips#inference#investment
OpenAI weighs chip alternatives after Nvidia inference gaps
technology5 months ago

OpenAI weighs chip alternatives after Nvidia inference gaps

OpenAI is reportedly seeking alternatives to Nvidia GPUs due to dissatisfaction with inference performance, citing eight sources. The move follows reports that Nvidia’s plan to invest up to $100 billion in OpenAI has stalled. OpenAI has previously struck deals with AMD and Broadcom to develop custom AI accelerators, signaling a push to diversify hardware sources even as Nvidia remains a major partner.

via Sherwood News|
#amd#broadcom#inference
Nvidia CEO Jensen Huang Envisions Unprecedented AI and Computing Growth
technology1 year ago

Nvidia CEO Jensen Huang Envisions Unprecedented AI and Computing Growth

Nvidia's CEO Jensen Huang addressed investor concerns about the company's future growth amid new AI model improvement methods like "test-time scaling," which enhances AI inference by adding more compute power. Despite competition from startups developing fast AI inference chips, Huang emphasized Nvidia's strong position in the market, noting that while most workloads currently focus on pretraining, the future will see increased AI inference. He reassured investors of Nvidia's scale and reliability, aligning with industry leaders like Microsoft's Satya Nadella on the significance of these developments.

via TechCrunch|
#ai#chips#inference
"Intel CEO Aims to Dethrone NVIDIA's CUDA Dominance, Open to Rival Chip Manufacturing"
technology2 years ago

"Intel CEO Aims to Dethrone NVIDIA's CUDA Dominance, Open to Rival Chip Manufacturing"

Intel's CEO, Pat Gelsinger, stated that the entire industry is motivated to eliminate NVIDIA's CUDA dominance in the AI market. Intel believes that the future of AI lies in inference rather than training models and aims to prioritize inference developments. Gelsinger sees NVIDIA's success as a temporary "bubble" and believes that the industry will adopt new training methods to bring a broader set of technologies. Intel praised its OpenVINO model and aims to transition towards next-gen markets. However, Intel needs to do more work to challenge CUDA's dominance, and for now, NVIDIA remains the leader in the AI segment.

via Wccftech|
#ai#cuda#inference