How to choose your Frontier Model
Model selection for your AI implementation
Choosing your Frontier Model
You’ve decided to consolidate your AI stack. But as models become platforms, the switching costs explode. A few lines of Python is now months of work and potentially tens or hundreds of thousands of dollars. How do you make your bet?
Before we get too far, I wanted to give a pointer to Tomasz Tunguz from Theory Ventures. His analysis of platforms, models, costs, inference is the best in the industry. If you want to go deep, I’d highly recommend following him on LinkedIn.
At Atmosphere, our main model is Claude, then we use ChatGPT for specific tasks, Perplexity and Gemini for search, and Grok for realtime.
We’re not covering the whole world of model delivery. Large platforms like Microsoft Azure, Amazon Bedrock and Salesforce offer plenty of capability and model choice, including homegrown models. Data platforms like Databricks and Snowflake offer AI tooling around enterprise data. Inference platforms like together.ai are great for Open Source.
We’ll keep this focused on the Frontier Labs.
Evaluation Criteria
You want to know what the model can do, how much will it cost for a given level of performance, what you can build on it, and if it meets your enterprise requirements.
Capabilities - domains of expertise (coding, finance, writing, analysis) and modalities (text, voice, image, video)
Economics - output speed (tokens/second) and price (in/out per 1m tokens)
Build Environment - agents, tooling, data connectors, SDKs, integrations
Enterprise Readiness - data security, SOC2, HIPAA, procurement, partner ecosystem
We’ve stopped paying attention to MMLU (Massive Multitask Language Understanding) scores. These are quantitative benchmarks that researchers used to demonstrate the capability of models. They don’t matter as much anymore because: i) the big labs keep leapfrogging each other., ii) MMLU doesn’t account for user interaction or engagement, and iii) smaller focused models can outperform the large ones at specific tasks.
What matters most if what you’re trying to do and your own eval.
How the Frontier Models stack up
This table is highly variable. I wrote the first draft a couple of weeks ago and essentially everything changed. Grok launched a very competitive voice model. All the pricing shifted.
To read the table, capabilities advance with every model release. Economics depend on the model choice; we’ve listed the low and high. Output speed is the same. Build environment is quickly evolving with more capability every week. The major platforms are reaching parity on enterprise adoption with friendlier terms such as no-training and ZDR (zero data retention) and better compliance.
The enterprise ecosystems are also expanding. OpenAI launched the OpenAI Deployment Company and Anthropic established a $1.5b joint venture for deployment. Every major consultancy now has a partnership with the big labs.
Open source
Cost has become a major issue with CEOs and CTOs looking to govern and rein in spending. A rogue agent can burn through thousands of dollars before someone notices.
The world of Open Source has shifted significantly in the last few months. While there were previously only 2-3 models (Llama, DeepSeek, Mistral) there are now ~10 serious contenders to evaluate.
Prices can be dramatically less expensive than the large models, roughly $0.05-$1.75 per 1m tokens input and $0.20-$4.40 per 1m tokens output.
That’s 5-20x cheaper than the higher end Frontier models.
But again the point isn’t about high-level cost, it’s about the specific thing you’re trying to do and whether the model you’re deploying is good at the actual problems you need to solve.
Open Source Models
As of writing the major Open Source models are:
DeepSeek (DeepSeek); strong reasoning at lower cost
Qwen (Alibaba): broad, general purpose
GLM (Zhipu AI): open-weight quality leader
Kimi (Moonshot AI): long-context and coding
MiniMax (MiniMax): efficient, agent-centric
Llama (Meta): broad capability
Mistral (Mistral): European leader
Gemma (Google): Google’s Open Source effort
GPT-OSS (OpenAI): cheap and efficient
You can run these models on 3rd party platforms like together.ai or on your own computer through a product like Ollama. If you’re going this route, you’re already at the leading edge of AI implementation.
To bring it all together, you have plenty of good options. And your evaluation should be tailored to the specific use case and workflows we mapped. That’s how to make your bet.
The last one in this series: the Agent Overview