Qualcomm Technologies announced its partnership with Mistral AI at the Snapdragon Summit 2024.
The collaboration aims to integrate Mistral AI’s generative AI models, Ministral 3B and Ministral 8B, into devices powered by Qualcomm’s Snapdragon 8 Elite Mobile Platform, Snapdragon Cockpit Elite, Snapdragon Ride Elite for automotive use, and Snapdragon X Series Compute Platforms for AI PCs.
Mistral AI’s Generative AI Models
Ministral 3B and 8B are Mistral AI’s first commercial AI models designed specifically for on-device use. These compact models offer significant power to devices like smartphones, PCs, and vehicles.
Running AI on-device delivers several benefits, including improved privacy, reduced latency, cost savings, enhanced reliability, and greater energy efficiency.
Key Features and Performance
These models excel in commonsense reasoning, knowledge, and task orchestration within the sub-10B parameter range. They are adaptable for a variety of applications, such as agent-based workflows and specialized task automation.
Both models support up to 128k context length, with the Ministral 8B model featuring a sliding-window attention pattern for faster and more efficient memory use during inference.
Mistral AI noted increasing demand from clients for local, privacy-first AI models capable of handling tasks like on-device translation, offline smart assistants, local analytics, and autonomous robotics.
The Ministral models meet this need by offering efficient, low-latency solutions. These models have demonstrated strong performance across various tasks, often surpassing competing models.
Functionality in Multi-Step Workflows
The models are also effective when used with larger language models like Mistral Large. They are optimized for function-calling in complex workflows, managing tasks such as input parsing, task routing, and executing API calls based on user intent.
This results in extremely low-latency responses and lower operational costs, making them suitable for real-world applications.
Qualcomm AI Hub Integration
Qualcomm and Mistral AI are working to make Ministral 3B and 8B available to developers soon via the Qualcomm AI Hub. Developers can already access other Mistral AI models, such as Mistral 7B v0.3, through the platform, empowering them to enhance their on-device AI solutions.
Speaking at the event, Durga Malladi, Senior Vice President and General Manager of Technology, Planning, and Edge Solutions at Qualcomm Technologies, Inc., said,
We are excited to announce our collaboration with Mistral AI, as it aligns with our goal of empowering edge devices with cutting-edge AI technology. Mistral AI’s Ministral 3B and Ministral 8B will enable device manufacturers, software vendors, and service providers to create innovative experiences, such as AI assistants and other applications that truly understand users’ needs, all while offering the immediacy, reliability, and enhanced privacy of on-device AI.
Speaking about the collaboration, Arthur Mensch, Co-founder and CEO of Mistral AI, said,
This collaboration with Qualcomm Technologies marks a major milestone for Mistral AI, showcasing how our new models can operate directly on devices powered by Snapdragon platforms. This enables faster local processing, which helps reduce both costs and energy consumption. Together, we are making advanced AI more accessible, empowering developers and users globally to experience the full potential of our models right on their devices.