tldrdailynews.com logo
Tag

Ai Inference

All articles tagged with #ai inference

Amazon-Qualcomm AI Deal Highlights Nvidia's AI-Chip Crown
technology1 month ago

Amazon-Qualcomm AI Deal Highlights Nvidia's AI-Chip Crown

Qualcomm inked a deal to supply Amazon with custom AI chips worth up to $60 billion over a decade, with a warrant for Amazon to buy up to 25 million Qualcomm shares based on purchases. The arrangement centers on AI inference, playing to Qualcomm’s efficiency and presenting a margin challenge rather than a direct threat to Nvidia’s leadership in AI training. Nvidia remains dominant in AI chips, reporting far larger data-center revenue, while Nvidia stock slipped slightly and Qualcomm rose on the news.

via Barron's|
#ai-chips#ai-inference#amazon
NVIDIA PAIR Turns Idle Home PCs Into a Private AI Cluster, Cutting Cloud Inference Bills
technology1 month ago

NVIDIA PAIR Turns Idle Home PCs Into a Private AI Cluster, Cutting Cloud Inference Bills

NVIDIA unveils PAIR, a software router that distributes AI inference across devices on a home network to create a private, cloud-free AI cluster. It auto-discovers machines, uses secure MTLS, and proxies via Ollama and LM Studio, with cross-platform support (Windows, Linux, macOS) and an open-source Apache 2.0 beta release. By routing idle GPU/CPU workloads to available nodes and avoiding apps like gaming on busy machines, PAIR aims to reduce cloud API usage—statements suggest substantial savings (e.g., around $1,200/month in cloud credits for certain workloads)—while keeping data and prompts local. It does not merge VRAM or tensor-split models; instead it uses a queue-based scheduler for multi-agent workflows, multi-tasking, and system offload, with planned future enhancements.

via Wccftech|
#ai-inference#edge-computing#home-cluster
Nvidia scales agentic AI performance with Groq 3 LPX in full production
technology1 month ago

Nvidia scales agentic AI performance with Groq 3 LPX in full production

Nvidia says its Groq 3 LPX dedicated AI inference accelerators are in full production, augmenting the Vera Rubin NVL72 platform to offload decode workloads and boost token-generation speed for complex agentic AI tasks. The system supports rack-scale deployments with up to 256 LP30 accelerators, aiming to deliver ultra-fast, real-time reasoning; Nebius is already deploying it via the Nebius Token Factory, reporting up to 3,400 tokens per second on the Gemma 4 31B model with a 100,000-token context, and SpaceX is also aligning its next-generation Vera Rubin-based AI architecture.

via SiliconANGLE|
#agentic-ai#ai-inference#groq-3-lpx
d-Matrix Unveils Raptor 3D-DRAM: A High-Bandwidth Path for Generative Inference
technology1 month ago

d-Matrix Unveils Raptor 3D-DRAM: A High-Bandwidth Path for Generative Inference

At Hot Chips 2026, d-Matrix unveils Raptor, a 3D-DRAM accelerator that stacks a logic die on a DRAM layer to deliver ultra-high bandwidth for generative AI inference. With a 1-Hi 32GB per card and a dense bank/channel layout, it uses stream-blocking and stream-flipping to achieve around 100 TB/s I/O and 0.37 pJ/bit, claiming ~32.6 GB/s per mm2 and 2.96 mW per GB/s versus HBM4. The system targets roughly 1,000 tokens/sec per user for frontier 3T-class models at 1M context, but faces intertwined challenges in bank-to-channel mapping, I/O power, and thermal reliability that will influence real-world viability.

via ServeTheHome|
#3d-dram#ai-inference#d-matrix
HBF Emerges as a NAND-Based Bridge to Supercharge AI Inference
technology2 months ago

HBF Emerges as a NAND-Based Bridge to Supercharge AI Inference

SK hynix and SanDisk unveiled High Bandwidth Flash (HBF), a NAND-based standard meant to sit between HBM and SSDs. Stacking NAND dies to up to 512GB per configuration, HBF delivers 0.4–3 TB/s bandwidth and is designed to expand AI inference memory by storing large models that are impractical for HBM, while signaling interoperability with CPUs/GPUs via the UCIe interconnect. It is positioned to complement, not replace, HBM and to foster an ecosystem of compatible products for faster AI workloads.

via Wccftech|
#ai-inference#high-bandwidth-flash#sandisk
AI Inference Gains a Near-Compute Memory Standard with HBF OCP Spec
technology2 months ago

AI Inference Gains a Near-Compute Memory Standard with HBF OCP Spec

SanDisk and SK hynix released the High Bandwidth Flash (HBF) technical specification through the Open Compute Project to standardize a near‑compute, high‑bandwidth memory solution for AI inference; with Google and Tenstorrent joining as consortia members, the spec defines system interfaces, electrical and packaging guidelines, reliability, and a software read/write guide, enabling HBF to coexist with High Bandwidth Memory and accelerating AI workloads. The spec is openly published to foster an early ecosystem, and Sandisk will discuss NAND’s role in AI at the FMS conference in Santa Clara.

via World Business Outlook|
#ai-inference#hbf#high-bandwidth-flash
Cerebras Delivers Breakneck LLM Speed, Yet Nvidia's CUDA Gravity Dominates
technology3 months ago

Cerebras Delivers Breakneck LLM Speed, Yet Nvidia's CUDA Gravity Dominates

NVIDIA just reported a blockbuster data-center quarter powered by CUDA, building a massive software moat, while Cerebras touts wafer-scale speed but braces for negative margins and heavy, CUDA‑centric integration that requires specialized compilation and custom engineering. Despite a $20B+ OpenAI inference deal and benchmarks showing ~21x latency advantage, the lack of broad framework support outside CUDA and the threat of OpenAI’s Jalapeño chip suggest Nvidia’s platform advantage remains hard to dethrone in the near term.

via 24/7 Wall St.|
#ai-inference#cerebras#cuda
Raymond James backs Nvidia ahead of Q1 with $323 target
market-news4 months ago

Raymond James backs Nvidia ahead of Q1 with $323 target

Raymond James keeps Nvidia (NVDA) at Strong Buy with a $323 target ahead of its May 21 Q1 report, citing a growing product roadmap and AI-inference ramp. The firm also notes Nvidia’s long-term outlook of roughly $1 trillion in cumulative GPU revenue through 2027 and believes the stock remains attractive on a roughly 18x 2027 P/E, with broader Street consensus implying about 22% upside.

via TipRanks|
#ai-inference#market-news#nvda
Nvidia Rolls Out Groq-Accelerated Inference Rack and Vera CPU Suite at GTC 2026
technology6 months ago

Nvidia Rolls Out Groq-Accelerated Inference Rack and Vera CPU Suite at GTC 2026

Nvidia announced at GTC 2026 a Groq-based 3 LPX inference rack (256 LPUs, liquid-cooled, Spectrum-X interconnect) paired with the Vera Rubin NVL72 rack, plus a Vera CPU rack and BlueField-4 STX storage rack, all aimed at accelerating trillion-parameter AI models and agentic AI workloads. Rubin CPX products were put on hold to focus on LPX, with availability in H2 2026 and OEMs like Dell, HPE and Oracle Cloud.

via crn.com|
#ai-inference#groq#lpu
Maia 200 Pushes Cloud AI In-House, But Nvidia Keeps the Data Center Edge
technology8 months ago

Maia 200 Pushes Cloud AI In-House, But Nvidia Keeps the Data Center Edge

Microsoft’s Maia 200 is an in‑house AI inference accelerator for Azure that claims strong performance per dollar and will power OpenAI models, signaling rising cloud‑provider pressure on Nvidia. While Maia 200 underscores a shift toward custom silicon, Nvidia still leads the data‑center AI market with its broad GPU ecosystem and software stack, and though cloud‑provider alternatives may erode pricing power over time, a rapid disruption to Nvidia’s position appears unlikely, even as valuations remain rich given AI growth.

via The Motley Fool|
#ai-inference#cloud-computing#data-center