OpenAI has announced GPT-4o mini (O for Omni), a compact yet powerful AI model aimed at making artificial intelligence more affordable and accessible.
It is priced at 15 cents per million input tokens and 60 cents per million output tokens, significantly cheaper than previous models like GPT-3.5 Turbo.
GPT-4o mini Features and Performance
- Performance Metrics: Scores 82% on the MMLU benchmark and leads the LMSYS leaderboard for chat preferences.
- Capabilities: Handles various tasks efficiently, making it suitable for applications needing multiple model calls, large context volumes, or real-time text interactions.
- Input Support: At the moment, it supports text and vision inputs, with future plans to add image, video, and audio inputs and outputs.
- Context Window: 128K tokens with up to 16K output tokens per request.
- Tokenizer Improvement: Enhanced for cost-effective handling of non-English text.
- Knowledge Base: Updated until October 2023.
Comparative Performance
- Textual and Multimodal Intelligence: Outperforms GPT-3.5 Turbo and other small models in academic benchmarks. Supports the same range of languages as GPT-4o.
- Function Calling: Enables developers to create applications that can fetch data or interact with external systems.
- Reasoning Tasks: Scores 82% on MMLU, compared to 77.9% for Gemini Flash and 73.8% for Claude Haiku.
- Math and Coding: Excels in mathematical reasoning and coding tasks, scoring 87% on MGSM and 87.2% on HumanEval.
- Multimodal Reasoning: Scores 59.4% on MMMU, outperforming Gemini Flash and Claude Haiku.
Partnerships and Applications
OpenAI collaborated with companies like Ramp and Superhuman to explore the model’s capabilities.
These partners found GPT-4o mini significantly better than GPT-3.5 Turbo for tasks like extracting structured data from receipts or generating high-quality email responses from thread history.
Safety Measures
OpenAI ensures safety is built into their models from the start, using methods like filtering out unwanted information during pre-training and reinforcement learning with human feedback (RLHF) during post-training.
GPT-4o mini has the same safety features as GPT-4o, tested by over 70 external experts in various fields. New techniques like the instruction hierarchy method improve the model’s resistance to jailbreaks, prompt injections, and system prompt extractions.
Future Plans
OpenAI aims to continue reducing costs while enhancing model capabilities. The cost per token of GPT-4o mini has dropped by 99% since the introduction of text-davinci-003 in 2022.
They envision AI models becoming seamlessly integrated into every app and website, making AI more accessible and embedded in daily digital experiences.
Availability and Pricing
- GPT-4o mini API Access: Available in the Assistants API, Chat Completions API, and Batch API at 15 cents per 1M input tokens and 60 cents per 1M output tokens.
- GPT-4o mini ChatGPT Access: Available to Free, Plus, and Team users starting today, replacing GPT-3.5. Enterprise users will gain access next week. Fine-tuning for GPT-4o mini will be available soon.