All articles tagged with #on device
Meta has paused its plan to charge a monthly subscription for the Conversation Focus feature on its Ray-Ban smart glasses, which runs on-device. While some premium features may eventually be subscription-based to subsidize hardware costs, Conversation Focus will remain free for now through an Early Access Program while Meta reevaluates the approach and the use of any rate limits.
PrismML released Bonsai 27B, a compressed, non-pretrained version of Qwen3.6-27B in two low-bit formats: ternary weights at 1.71 bits/weight (~5.9GB) and 1-bit weights at 1.125 bits/weight (~3.9GB). Both variants are multimodal and maintain the same architecture, with memory and KV cache constraints shaping device viability. In benchmarking across 15 tests on H100, ternary Bonsai preserves about 94.6% and 1-bit about 89.5% of FP16 accuracy, and with additional 4-bit KV cache it can fit on devices; PrismML notes on-device usage for laptops and phones, speedups via DSpark, and Apache 2.0 licensing with llama.cpp/MLX support.
iOS 27 overhauls Siri into Siri AI, a capable, context-aware assistant that can search across Mail, Messages, Photos, and more, pull in broad web knowledge, analyze images, and perform tasks inside apps using App Actions. It introduces a new Search and Ask Siri interface on iPhone, a dedicated Siri app, and Write with Siri for drafting and editing text. An on‑device voice model (customizable on compatible iPhone models) improves dictation and speech understanding, with privacy maintained via Private Cloud Compute. Siri AI works across iOS 27, iPadOS 27, macOS Golden Gate, visionOS 27, and watchOS 27, syncing conversations across devices, and will roll out this fall to devices meeting the requirements (iPhone 15 Pro or newer), with initial regional availability limited in EU/China.
Apple unveils AFM 3, its third generation of Foundation Models, a five-model lineup spanning on-device options (AFM 3 Core and AFM 3 Core Advanced) and cloud-based ones (AFM 3 Cloud, ADM 3 Cloud (Image), AFM 3 Cloud Pro), with AFM 3 Cloud Pro hosted on NVIDIA GPUs in Google Cloud. The rollout extends Private Cloud Compute to third-party infrastructure while preserving Apple’s privacy protections, highlights a sparse 20‑billion-parameter AFM 3 Core Advanced architecture, and reports extensive human evaluations showing improvements over prior generations; training used a mix of public, licensed, open-source, synthetic data and did not include user data, with publishers able to opt out of training.
Google releases AI Edge Gallery for Mac, letting users run Gemma 4 12B and other Gemma models locally offline with on‑device dictation via AI Edge Eloquent; the setup requires about 16GB of VRAM/unified memory, making most modern Macs eligible (except the MacBook Neo). Gemma is an open, portable family distinct from Google’s private Gemini models, and the app offers offline speed and privacy with a direct download from Google.
Apple plans to foreground AI that runs on iPhones and other devices at WWDC, leveraging its 15 years of silicon design to speed edge AI. However, many queries will still require cloud processing for complexity, with Siri tasks routing to Google Cloud’s Gemini and Nvidia privacy tech used in that setting, signaling a hybrid approach that boosts local AI while maintaining cloud-backed capabilities.
AI browsers are poised to become orchestration hubs that run agentic AI and route workloads across local devices, enterprise data centers, and the cloud, using local models and cross-environment coordination to improve efficiency, privacy, and responsiveness—though security concerns and deployment timelines remain.
Clawdbot is an open-source AI personal assistant that runs locally on your device (often a Mac mini), can read/write files, access your email/calendar/docs, and act autonomously on your behalf; it has sparked Silicon Valley buzz but requires technical setup and carries notable security and privacy risks, with installation and security tools available on GitHub for Mac, Windows, and Linux.