All articles tagged with #multimodal
Proton unveils Lumo 2.0, a privacy-first AI assistant with image recognition/generation, user-controlled memory, encrypted projects, and live web search; Lumo for Business adds zero-access encryption and European infrastructure for enterprise use, while performance benchmarks show substantial gains over prior versions. The update includes free core features and two paid plans, Lumo Plus and Lumo Professional.
At WWDC, Apple announced iOS 27 will relocate Siri into its own standalone app, rebuilt with Apple Intelligence for a more conversational, context-aware experience with improved dictation, image understanding, and cross-device recall, while preserving quick access via the side button and expanding support to newer iPhone models.
Google has brought its AI Edge Gallery to macOS, giving Mac users access to five Google-owned models led by Gemma 4 12B, a 12‑billion‑parameter multimodal model designed to run locally on laptops with 16GB RAM. The gallery also includes Gemma-4-E2B-it, Gemma-4-E4B-it, Gemma-3n-E2B-it, and Gemma-3n-E4B-it, all accessible offline and designed for private on-device use. In tandem, Google released the Google AI Edge Eloquent dictation app for Mac, which transcribes speech on-device, refines text, supports writing styles, and lets users add custom vocabulary for better accuracy.
Google unveils Gemma 4 12B, a 12-billion-parameter model designed to run on consumer laptops with 16GB of RAM or VRAM. It achieves near-26B performance thanks to Multi-Token Prediction (MTP) and a streamlined, memory-efficient multimodal pipeline that handles text, audio, and images without a bulky encoder. The model is intended for on-device use and can be downloaded from Kaggle or Hugging Face, with integration via tools like LM Studio and Google AI Edge Gallery.
NVIDIA unveiled Cosmos 3, a fully open omnimodel designed for physical AI that combines a reasoning transformer with an expert generation transformer to understand object interactions, simulate environments, and generate actions across text, images, video, ambient sound and motion. Marketed as the first open omnimodel of its kind, it supports synthetic data generation and serves as a vision-language model, a world/simulation model, and an action-policy backbone for robotics and autonomous vehicles. The rollout includes the Cosmos Coalition with partners like Agile Robots, Black Forest Labs, Generalist, LTX, Runway and Skild AI, and Cosmos 3 is available in Super and Nano now, with Edge coming soon; access is via NVIDIA and collaborators such as Hugging Face and Azure.
Google unveiled a sweeping AI-powered redesign of Search, adding a longer, more conversational interface, multimodal inputs (photos, videos, documents), and autonomous “agents” built on Gemini 3.5 Flash to monitor topics automatically. A new Gemini Spark integrates AI into Gmail and Docs, while a revamped shopping experience surfaces discounts. The changes aim to boost usage and ad-targeting but raise concerns about transparency, user choice, and potential reductions in traffic to third-party sites, fueling debate about the future of the open web.
Google introduces Gemini Omni, a new multi-modal AI world-model family led by Gemini Omni Flash that can accept text, audio, images, and video to generate and edit realistic videos with improved physics; it supports avatars and interactive edits via conversation, with outputs watermarked by SynthID for AI verification, and will roll out to paid Google AI plans before expanding to YouTube Shorts and YouTube Create.
Google’s Gemma 4 proves you don’t need a giant parameter count to punch above your weight: with 2.3 billion parameters it rivals 70B-scale models in performance while running offline on under 1.5 GB of RAM. The model is designed for edge use, features a 128K context window, and supports text, vision, and audio inputs in a multimodal architecture under an Apache 2.0 open-source license. It also handles 140 languages and emphasizes privacy and offline operation, making it suitable for resource-constrained devices. Notable caveats include coding task limitations and incomplete iOS bindings, with ongoing community-driven improvements anticipated.
Google’s Gemma 4 is an open-source, locally installable multimodal AI that runs on smartphones and laptops without needing cloud processing, prioritizing privacy and cost efficiency. It comes in dense (31B parameters) and sparse (26B parameters) architectures using a mixture-of-experts approach to balance performance and efficiency. Capable of text, image, and audio processing, it targets coding, creative writing, UI design, healthcare and education, with deployment support through LM Studio, Ollama, Llama CPP and Supabase. Four device-optimized versions offer offline functionality and reduced reliance on cloud services, pushing accessible, private AI for areas with limited connectivity.
Meta unveiled Muse Spark, its first model in a new Muse series, to run in the Meta AI app and website in the US and roll out to WhatsApp, Instagram, Facebook, Messenger, and Meta’s smart glasses. The multimodal, multi‑agent system will be available to partners via API, supports text+image input, and offers Instant and Thinking modes for faster vs. deeper responses. Meta says Muse Spark can tackle complex science, math, and health questions and aims to compete with OpenAI’s ChatGPT Health and Claude, with future versions planned to be open-sourced as the company expands its AI efforts after earlier Llama initiatives.
Meta unveiled Muse Spark, its first AI model developed under Alexandr Wang, aiming to narrow the gap with OpenAI and Anthropic. The model will power queries in the Meta AI app and Meta.ai, with plans to expand to Facebook, Instagram, and WhatsApp, and accepts voice, text, and image inputs but outputs text. It includes modes like a fast query mode and a 'shopping mode' tied to user data; an open-source version is planned. While competitive on several tasks, it lags in areas like coding. All flavors are free but may be rate-limited, and Meta’s privacy policy indicates broad data use for AI features.
Google's Gemini 3.1 Pro is pitched as a versatile, multi-modal AI capable of long-context, multi-step reasoning. The article tests its capabilities and presents seven concrete prompts—long-document extraction, logic stress testing, code architecture, getting unstuck fast, contextual video analysis, synthetic data generation, and deep research synthesis—to turn the model into an execution engine for productive, deep work.
Apple researchers have developed UniGen-1.5, a unified AI model capable of understanding, generating, and editing images within a single system, enhancing previous models with new editing capabilities and improved instruction alignment, achieving state-of-the-art performance on various benchmarks, though it still faces challenges with text generation and identity consistency.
Apple has developed two advanced multilingual, multimodal foundation language models: a 3-billion-parameter on-device model optimized for Apple silicon and a scalable server model using a novel PT-MoE transformer, both supporting multiple languages and image understanding. These models power Apple Intelligence features across devices and services, with a focus on responsible AI, privacy, and developer integration through a new Swift-based framework. They outperform comparable open models in benchmarks and human evaluations, enhancing user experiences with efficient, accurate, and responsible AI capabilities.
Google is expanding its AI Mode feature to India, marking its first international launch after debuting in the US. Powered by Gemini 2.5, AI Mode enhances search with advanced reasoning, multimodal inputs including voice and images, and real-time data sources. Users in India can now access AI Mode via the Google app or web, enabling more interactive and detailed search experiences.