Tag

Ai Inference

All articles tagged with #ai inference

technology14 days ago•21 min saved

Cerebras Delivers Breakneck LLM Speed, Yet Nvidia's CUDA Gravity Dominates

NVIDIA just reported a blockbuster data-center quarter powered by CUDA, building a massive software moat, while Cerebras touts wafer-scale speed but braces for negative margins and heavy, CUDA‑centric integration that requires specialized compilation and custom engineering. Despite a $20B+ OpenAI inference deal and benchmarks showing ~21x latency advantage, the lack of broad framework support outside CUDA and the threat of OpenAI’s Jalapeño chip suggest Nvidia’s platform advantage remains hard to dethrone in the near term.

via 24/7 Wall St.|

#ai-inference #cerebras #cuda

ai17 days ago•31 min saved

OpenAI unveils Jalapeño, its first custom AI processor for inference

OpenAI, with Broadcom, unveils Jalapeño, its first AI inference ASIC for servers—designed to power current and future models with better performance per watt and less reliance on Nvidia GPUs, and dubbed the first step in a multi-generation compute platform set for deployment by the end of 2026.

via The Verge|

#ai #ai-chips #ai-inference

market-news1 month ago•15 min saved

Raymond James backs Nvidia ahead of Q1 with $323 target

Raymond James keeps Nvidia (NVDA) at Strong Buy with a $323 target ahead of its May 21 Q1 report, citing a growing product roadmap and AI-inference ramp. The firm also notes Nvidia’s long-term outlook of roughly $1 trillion in cumulative GPU revenue through 2027 and believes the stock remains attractive on a roughly 18x 2027 P/E, with broader Street consensus implying about 22% upside.

via TipRanks|

#ai-inference #market-news #nvda

technology3 months ago•3 min saved

Arm's In-House AGI Chip Ignites Big Revenue Outlook, Lifts Shares

Arm stock rose about 16% after CEO Rene Haas said its new AGI CPU designed for AI inference in data centers could generate about $15 billion in annual revenue by 2031, with total revenue around $25 billion and EPS of $9; first customers include Meta, OpenAI, Cloudflare, and SAP, signaling a historic pivot toward in-house chip manufacturing for Arm.

via CNBC|

#agi-cpu #ai-inference #arm

technology3 months ago•10 min saved

Nvidia Rolls Out Groq-Accelerated Inference Rack and Vera CPU Suite at GTC 2026

Nvidia announced at GTC 2026 a Groq-based 3 LPX inference rack (256 LPUs, liquid-cooled, Spectrum-X interconnect) paired with the Vera Rubin NVL72 rack, plus a Vera CPU rack and BlueField-4 STX storage rack, all aimed at accelerating trillion-parameter AI models and agentic AI workloads. Rubin CPX products were put on hold to focus on LPX, with availability in H2 2026 and OEMs like Dell, HPE and Oracle Cloud.

via crn.com|

#ai-inference #groq #lpu

technology5 months ago•7 min saved

Maia 200 Pushes Cloud AI In-House, But Nvidia Keeps the Data Center Edge

Microsoft’s Maia 200 is an in‑house AI inference accelerator for Azure that claims strong performance per dollar and will power OpenAI models, signaling rising cloud‑provider pressure on Nvidia. While Maia 200 underscores a shift toward custom silicon, Nvidia still leads the data‑center AI market with its broad GPU ecosystem and software stack, and though cloud‑provider alternatives may erode pricing power over time, a rapid disruption to Nvidia’s position appears unlikely, even as valuations remain rich given AI growth.

via The Motley Fool|

#ai-inference #cloud-computing #data-center

technology6 months ago•2 min saved

Nvidia's $20B AI Inference Bet Boosts Groq and Reshapes the AI Landscape

Nvidia's partnership with Groq, focusing on inference technology, highlights the importance of efficient AI inference in scaling AI applications, potentially giving Nvidia an edge in the AI race by accelerating and reducing the cost of deploying large language models.

via Axios|

#ai-hardware #ai-inference #groq

business6 months ago•3 min saved

Nvidia's Strategic Partnership with Groq Boosts AI Chip Competition and Stock

Nvidia's strategic licensing agreement with AI startup Groq, including key personnel hires, aims to strengthen its position in AI inference technology, signaling a shift from training to inference workloads and potentially expanding Nvidia's market dominance. The deal, which keeps Groq independent, is viewed positively by analysts as a move to address market share concerns and diversify Nvidia's AI offerings.

via Investor's Business Daily|

#ai-chips #ai-inference #business

technology9 months ago•2 min saved

Intel Unveils Crescent Island GPU with 160GB Memory for AI Inference

Intel has announced the Crescent Island GPU for data centers, featuring the new Xe3P architecture, 160 GB LPDDR5X memory, and optimized for AI inference workloads, with customer sampling expected in the second half of 2026.

via Wccftech|

#ai-inference #crescent-island #gpu

technology9 months ago•4 min saved

Intel Unveils Crescent Island Inference GPU with 160GB vRAM

Intel announced Crescent Island, a next-generation inference-optimized enterprise GPU built on the Xe3P architecture with 160GB of LPDDR5x memory, targeting AI inference workloads with a focus on power efficiency and cost, but it won't be available for sampling until H2 2026 at the earliest, with broad shipping likely in 2027.

via Phoronix|

#ai-inference #crescent-island #gpu

technology9 months ago•3 min saved

oLLM: Lightweight Python Library Enables 100K-Context LLMs on 8GB GPUs with SSD Offload

oLLM is a lightweight Python library that enables large-context LLM inference on consumer GPUs by offloading weights and KV-cache to SSDs, maintaining high precision without quantization, and supporting models like Qwen3-Next-80B, GPT-OSS-20B, and Llama-3, making it feasible to run large models on 8 GB GPUs for offline tasks, though with lower throughput and storage demands.

via MarkTechPost|

#ai-inference #gpu-memory-optimization #large-language-model

technology10 months ago•5 min saved

NVIDIA Launches Rubin CPX: A Next-Gen AI GPU for Video and Software Innovation

NVIDIA has announced the Rubin CPX, a new GPU designed for massive-context AI applications like long-format video and large-scale software coding, offering unprecedented performance with 8 exaflops, 100TB memory, and integration with the Vera Rubin platform, enabling significant advancements in AI productivity and monetization.

via NVIDIA Newsroom|

#ai-inference #gpu #massive-context-processing

technology10 months ago•6 min saved

Baseten Secures $150M in Funding, Valuation Hits $2.15B

Baseten, an AI inference infrastructure startup, raised $150 million at a $2.15 billion valuation, reflecting rapid growth in AI deployment tools, with a focus on inference as a key market in AI development.

via Fortune|

#ai-inference #ai-infrastructure #baseten

technology10 months ago•54 min saved

Microsoft's Analog Optical Computer Boosts AI and Optimization

The article introduces the analog optical computer (AOC), a novel hardware platform that combines optical and electronic components to efficiently perform AI inference and combinatorial optimization tasks through fixed-point iterative processes, promising significant improvements in speed and energy efficiency over digital systems.

via Nature|

#ai-inference #analog-computing #combinatorial-optimization

technology2 years ago•6 min saved

"Groq's LPU: The Future Standard for AI Startups' Speedy Computation"

Groq, a Silicon Valley-based company, is making waves in the AI chip race with its language processing units (LPUs) designed for AI language applications. CEO Jonathan Ross claims that most startups will be using Groq's LPUs by the end of 2024 due to their super-fast and cost-effective performance for large language model (LLM) inference. Ross also highlighted the advantages of Groq's LPUs over Nvidia GPUs, emphasizing their ability to provide faster LLM output and maintain privacy in chat queries. The company has seen a surge in interest following a viral moment and is poised to contribute to the supply of AI chips, with plans to increase capacity and collaborate with countries.

via VentureBeat|

#ai-chips #ai-inference #groq