Multimodal Models: OpenAI, Anthropic, and DeepSeek Race to Lead AI Performance

Artificial intelligence is moving beyond the question of whether a system can generate fluent text. Leading models are increasingly expected to interpret photographs, diagrams, audio, video, documents, software interfaces, and physical surroundings, then connect those signals to reasoning and action. This shift has created a high-stakes contest among OpenAI, Anthropic, and DeepSeek.

The competition is not simply about a higher score on a single benchmark. It involves latency, accuracy, tool use, cost, openness, safety, developer access, and the ability to sustain reliable performance across long workflows. A model that recognizes an image but cannot use that information consistently in an agent task may be less valuable than a slightly weaker model with better memory, lower cost, or more dependable execution.

Multimodal AI refers to systems that process or generate more than one type of information, such as text, images, audio, and video, within connected tasks.

Three different strategies

OpenAI established a strong public reference point with GPT-4o, introduced in May 2024. OpenAI described the model as capable of accepting combinations of text, audio, image, and video inputs, while producing text, audio, and image outputs. The company also reported an average audio response time of 320 milliseconds, with a minimum of 232 milliseconds, making conversation feel closer to a natural exchange.[1]

GPT-4o also represented a technical choice: OpenAI said that text, vision, and audio were trained in a single end-to-end model rather than being handled by a chain of separate systems. That design can preserve information such as tone, background sound, and visual context across a conversation. OpenAI reported GPT-4 Turbo-level results on English text and code, improved performance in several non-English languages, and a lower API price at launch.[1]

Anthropic has taken a somewhat different route. Claude supports image understanding through its consumer interface, developer console, and API. Its documentation allows images to be supplied as base64 data, web URLs, or reusable file identifiers.[2] The company has placed particular emphasis on careful instruction following, extended reasoning, coding, tool use, and agent workflows. In Claude 4, Opus and Sonnet models can use tools during extended reasoning, operate tools in parallel, and work with files and external systems.[3]

Anthropic’s own reported results show why multimodal performance should not be reduced to image recognition alone. In its Claude 4 announcement, the company reported 73.7% for Opus 4 and 72.6% for Sonnet 4 on the MMMU benchmark when extended reasoning was used. On MMMLU, it reported 87.4% and 85.4%, respectively.[3] These are vendor-reported figures, and comparisons require caution because prompts, inference budgets, scaffolds, and evaluation procedures can differ.

DeepSeek has introduced a more cost-focused and open-development challenge. On August 21, 2026, DeepSeek announced DeepSeek-V4-Flash-Vision-Exp, an experimental multimodal model available through its API.[4] DeepSeek says the model preserves V4-Flash performance on text tasks, including reasoning and agent work, while making a substantial improvement on multimodal agent benchmarks. The company also says that its results approach the performance of Opus-4.8.[4] That claim should be treated as an announcement from the vendor until independent testing confirms its scope.

DeepSeek’s API design illustrates its practical priorities. Mixed text and image requests are supported through base64 data, external URLs, or the Files API. Images are tokenized for billing, and the service allows developers to upload an image once and reuse it across requests.[4] These details matter to companies building document-processing, inspection, customer-support, and automation systems where repeated image handling can become expensive.

Comparison at a glance

Dimension	OpenAI	Anthropic	DeepSeek
Core multimodal emphasis	Real-time text, vision, audio, and video interaction	Image understanding linked to reasoning, coding, tools, and agents	Cost-efficient vision plus text reasoning and multimodal agents
Representative public evidence	GPT-4o accepts mixed text, image, audio, and video inputs	Claude 4 reports strong results on MMMU and MMMLU with extended reasoning	V4-Flash-Vision-Exp claims performance close to Opus-4.8 on multimodal agent benchmarks
Developer experience	Broad product and API ecosystem with real-time interaction	API, console, files, tool use, and enterprise cloud distribution	OpenAI-compatible API style, image URLs, base64, and reusable file IDs
Key competitive advantage	Low-latency natural interaction and broad modality coverage	Sustained reasoning and reliable agent workflows	Price sensitivity, fast iteration, and access to experimental models
Main uncertainty	Consistency across every modality and deployment setting	Audio and video coverage compared with native omni systems	Independent validation and long-term production reliability

What “performance” now means

A useful multimodal evaluation must test several layers at once. First comes perception: can a model read small text, interpret a chart, identify objects, distinguish speakers, or follow movement in a video? Second comes grounding: can it connect what it sees or hears to the user’s actual question? Third comes reasoning: can it combine visual evidence with prior instructions and reach a defensible conclusion? Fourth comes action: can it call a tool, edit a file, query a database, or control a workflow without losing the relevant context?

These layers create trade-offs. A model may perform well on static images but struggle with long video. Another may solve visual puzzles but hallucinate details when the image is blurry. A third may understand a document yet fail to cite the correct page when operating inside a retrieval system. The best product therefore depends on the task, not only on a leaderboard position.

OpenAI’s GPT-4o demonstrates the value of speed. Fast audio responses can support tutoring, accessibility, translation, customer service, and hands-free assistance. Anthropic’s strategy is especially compelling for professional work in which the model must inspect files, reason for longer periods, use tools, and maintain coherence across many steps. DeepSeek’s appeal is strongest where organizations need lower operating costs, API flexibility, and rapid access to new model variants.

 

The role of cost and infrastructure

Multimodal inference is more demanding than text-only inference because images, audio segments, and video frames must be encoded and carried through the model’s context. Costs also rise when an agent repeatedly reviews the same visual evidence or produces several candidate actions. This makes tokenization, caching, context management, and hardware efficiency central competitive factors.

DeepSeek’s announcement highlights image-token accounting and reusable files, while OpenAI has promoted API efficiency and lower pricing for GPT-4o relative to GPT-4 Turbo at launch.[1][4] Anthropic’s Files API and prompt caching capabilities address a similar production concern: developers need to avoid sending the same large inputs unnecessarily.[3] The contest is therefore partly an infrastructure contest. Better model quality can be cancelled by excessive latency or an unsustainable bill.

Safety and trust

Adding vision, audio, and video creates risks that do not appear in the same form in text-only systems. Models can infer sensitive attributes, imitate voices, expose private information in photographs, or misread a scene in a way that causes physical or financial harm. Audio systems also need protections against impersonation and emotional manipulation.

OpenAI reported safety evaluations and external red teaming for GPT-4o, including work on bias, fairness, social psychology, misinformation, and new audio risks.[1] Anthropic has tied Claude 4’s agent capabilities to safety testing and higher safety-level protections.[3] DeepSeek’s experimental release expands access to multimodal agents, but public technical and safety documentation will be important for organizations deciding whether to use it in sensitive environments.[4]

Safety is not separate from performance. A system that refuses unsafe requests appropriately, identifies uncertainty, preserves privacy, and asks for confirmation before consequential actions is more useful in serious deployments than one that merely produces impressive demonstrations.

Figure 2. Reference diagram concerning DeepSeek vision-model research. Image source: Roboflow.

Who is best positioned?

OpenAI currently has a strong claim in real-time, consumer-facing multimodal interaction. Its integration of audio and vision into a widely known product gives it a powerful distribution advantage. Anthropic is highly competitive in multimodal workflows that depend on long reasoning, coding, file use, and agents. DeepSeek is applying pressure through cost, fast release cycles, API accessibility, and claims of narrowing the gap on multimodal agent tasks.

No single ranking captures all three positions. OpenAI may be the better choice for live voice interaction; Anthropic may be preferable for a long software-engineering session; DeepSeek may offer attractive economics for high-volume visual processing. Buyers should test representative tasks with identical prompts, the same context, comparable reasoning budgets, and a clear measure of error cost.

The contest among OpenAI, Anthropic, and DeepSeek is changing the meaning of a leading AI model. Success now requires more than fluent answers. It requires accurate perception, grounded reasoning, useful actions, predictable latency, manageable cost, and safeguards appropriate to the setting.

OpenAI’s omni design has made real-time interaction a central product benchmark. Anthropic has linked vision to sustained reasoning and agent reliability. DeepSeek is challenging established pricing and access assumptions while pushing experimental multimodal agents into public APIs. The next decisive advantage may belong not to the company with the strongest isolated score, but to the developer that combines model quality, infrastructure, safety, and distribution into a dependable system.

References

1. OpenAI, “Hello GPT-4o,” May 13, 2024.

2. Anthropic, “Vision — Claude Platform Documentation”.

3. Anthropic, “Introducing Claude 4,” May 22, 2025.

4. DeepSeek API Docs, “DeepSeek-V4-Flash-Vision-Exp Release: Multimodal API Now Live,” August 21, 2026.

5. Microsoft Azure, “Introducing GPT-4o: OpenAI’s New Flagship Multimodal Model,” May 2024.

6. Roboflow, “DeepSeek Vision Models: Janus, VL2, and OCR”

Reference videos

The following videos provide visual context for the competition. They are demonstrations, not neutral scientific evaluations, so viewers should distinguish product presentation from independent testing.

1. OpenAI: “Say hello to GPT-4o” — a launch presentation showing text, vision, and audio interaction.

2. OpenAI: GPT-4o vision demonstration — a focused example of visual reasoning in a live setting.

3. Anthropic: Introducing Claude 4 — the company’s announcement covering reasoning, tools, coding, and agent workflows.

4. DeepSeek: V4-Flash-Vision-Exp release — the API announcement for DeepSeek’s experimental multimodal model.