Tag

Local Inference

All articles tagged with #local inference

technology1 month ago•6 min saved

AMD Strix Halo Brings 120B-Parameter AI to a $1,500 PC

AMD's Strix Halo Ryzen AI Max Plus enables on-device inference for models up to 120B parameters using up to 128 GB of unified memory in a $1,499 mini PC, offering strong cost efficiency and data control but with real-world memory bandwidth around 122 GB/s and a still-maturing ROCm/Windows ecosystem. It trails NVIDIA in prefill speed and lacks CUDA-level software maturity, though its cost per GB memory (~$25.77) is far cheaper than Apple M3 Ultra’s. A future Gorgon Halo with 192 GB and up to 300B parameters is planned for 2026, signaling AMD's ongoing AI hardware push. For workloads prioritizing local processing and agent workflows, Strix Halo is compelling; for CUDA-heavy or bandwidth-intensive tasks, NVIDIA or Apple may be preferable.

via Geeky Gadgets|

#ai-hardware #amd #cost-efficiency

technology3 months ago•3 min saved

Ubuntu's AI Plan: Snap-Delivered, Opt-In Features with Local Inference

Canonical says Ubuntu's upcoming AI features will be delivered via Snap packages and start as a strictly opt-in preview in 26.10, with no global AI kill switch—users can disable AI by removing the relevant Snaps; future releases will add a setup wizard to select preferred AI features, all using local inference rather than cloud services, with examples like text-to-speech and camera focus, and some code may be authored with AI.

via Phoronix|

#ai #linux #local-inference

technology4 months ago•2 min saved

Ubuntu to Roll Out AI Features With Local Inference Throughout 2026

Canonical says Ubuntu will add AI features over 2026, starting after the 26.04 LTS release, with a focus on secure, locally run AI that enables a context‑aware OS. The plan includes integrating agentic workflows for desktop and server use, exploring AI for tasks like interpreting system logs, and providing frontier AI access in a way that respects open‑source values. Emphasis is on efficient local inference, robust education for engineers, silicon partnerships, and not turning Ubuntu into an AI product, but making the OS significantly stronger through thoughtful AI integration.

via Phoronix|

#ai #canonical #local-inference