Is AI Getting Cheaper? How to Measure LLM Cost Per Task and Per Intelligence Level
Melvin Vivas · X video post · 2026-10-14 · 18:17 · 598 views · Open on X
Topics: LLM Fundamentals, LLMOps, Deployment & Monitoring, Industry Trends & Job Market · Level: intermediate
Summary
Melvin Vivas shares a video by Theo (t3.gg). Theo argues that AI is getting cheaper even though AI bills keep rising, and that price per token is the wrong way to measure it. He breaks cost into three parts: price per token, tokens used per task, and cost to reach a given level of intelligence. Using Artificial Analysis data, he shows that the cost of a fixed intelligence level drops by far more than 10x a year. Bills go up because people use models much more (Jevons paradox), not because intelligence got more expensive.
Key points
- Price per token alone is misleading. Look at three things: (1) price per token, (2) tokens needed to finish a task, and (3) cost to reach a given intelligence level.
- Top models did get pricier per token: GPT-5 medium was $1.25 in / $10 out per million tokens, while GPT-6 Astra is $10 in / $50 out. Cheap models also improved, though: GLM 5.3 Flash is about $0.15 in / $0.50 out and beats GPT-5 at agentic work.
- Token-hungry models eat up their price advantage. Gemini 3.8 Flash is more than 10x cheaper per token than Astra, but its cost per task is less than 3x cheaper. Sonnet 5 cost more per task than Astra because it burns tokens.
- Pick models by workload. For jobs heavy on input (for example, categorizing 800K tokens), choose the cheapest input-token price. For agentic coding, caching, reliability and token efficiency matter more.
- Compare models with a cost-vs-intelligence chart and its Pareto frontier. Going from a score of about 45 ($0.82 per task) to the 50s ($3.26) triples the cost for a few points.
- Hold intelligence fixed and look at cost over time. DeepSeek V4 Flash roughly matches GPT-5 for about 20x less. o1 Preview's level cost about $15 per task in Sept 2024, and Grok 3 Mini beat it for about $0.16 by Feb 2025. Each price drop comes faster than the last.
- Bills rise because of Jevons paradox: cheaper, more capable models run longer, end-to-end agentic tasks, so total usage and spend go up.
- Small models with good tool use (search, tool calls, long-running agentic work) can beat last year's frontier models, even when they hold less built-in knowledge.
Resources mentioned
- Artificial Analysis · website · artificialanalysis.ai · free
Independent benchmarking site that compares LLMs on intelligence index, price, cost per task and the cost-vs-intelligence Pareto frontier.
Also in: Liquid AI releases d1-ultra decision model (Melvin Vivas on X · notes), ElevenLabs Eleven v4 & v4 Turbo: Emotive, Multilingual Text-to-Speech Models (Melvin Vivas on X · notes), Be Skeptical of Model Leaderboards: Muse Spark vs Astra (Melvin Vivas on X · notes), Pipette: Open-Source Benchmarking for On-Device Models (Melvin Vivas on X · notes) and 3 more - Railway · tool · railway.com · free · recommended by both Bashiri Smith & Melvin Vivas
Deploy apps and APIs quickly.
Also in: Railway Platform Incident: Check Provider Status Pages (Melvin Vivas on X · notes) - GLM 5.3 Flash · tool · huggingface.co · free
Low-cost model (about $0.15 in / $0.50 out per million tokens) that Theo says beats GPT-5 at agentic work.
Also in: GLM-Legal: Z.ai's Legal-Domain Model Built on GLM-5.3-Flash (Melvin Vivas on X · notes), Coworker Desktop App Works with Any OpenAI-Compatible Endpoint (Melvin Vivas on X · notes), Running GLM 5.3 Flash on Baseten with the Pi Coding Agent (Melvin Vivas on X · notes) - DeepSeek V4 Flash · tool · huggingface.co · free
Cheap DeepSeek model whose intelligence-index score is roughly equal to GPT-5's.
Also in: Low-Cost Agent Run: DeepSeek V4 Flash via OpenRouter in ohmypi (Melvin Vivas on X · notes), Running Codex with DeepSeek V4 Flash through OpenRouter (Melvin Vivas on X · notes), LFM2.5-2.6B Matches DeepSeek-V4-Flash on Tool Calling; LEAP Fine-Tuning (Melvin Vivas on X · notes), DeepSeek V4 Flash at 90% Off on Nous Portal (Melvin Vivas on X · notes) and 4 more - Gemini 3.8 Flash · tool · deepmind.google · free
Google's fast, low-cost Gemini model, suited to auxiliary tasks like web browsing and vision.
Also in: Testing Gemini 3.8 Flash in Cursor with a CRM Smoke Test (Melvin Vivas on X · notes), Gemini 3.8 Flash Available in Cursor CLI (Melvin Vivas on X · notes), Models That Work With the Hermes Agent for Personal Productivity (Melvin Vivas on X · notes), Cut Hermes Agent Token Costs with Gemini Flash as an Auxiliary Model (Melvin Vivas on X · notes) - GPT-6 Astra · tool · openai.com · paid
The model announced in the quoted launch post, pitched as the developer's most capable model for work, coding, science and cybersecurity, and able to operate a computer.
Also in: Glimpse: A Visual Reasoning Model That Uses a Visual Scratch Space (Melvin Vivas on X · notes), GitHub Copilot Adds Intelligent Local Model Routing (Project HydraFusion) (Melvin Vivas on X · notes), Comparing Opus 5.5, GPT-6 Astra and Grok 4.7 on OpenRouter (Melvin Vivas on X · notes), OpenAI Makes GPT-6 Astra and GPT-6.1 Sol About 50% Faster (Melvin Vivas on X · notes) and 55 more - Fable · tool · anthropic.com · paid
Named as what the creator used with Devin for this build; the post gives no details about what it is.
Also in: Demo: One-Shotting a Flappy Bird iPhone App with Devin and Fable 5.1 (Melvin Vivas on X · notes), Subagents with mixed models: a strong planner and a fast executor (Melvin Vivas on X · notes), Cognition's SWE-2 Coding Model Now in Devin (Melvin Vivas on X · notes), GPT-6 Astra Access Across Plans vs Fable on Claude Max (Melvin Vivas on X · notes) and 11 more - Claude Sonnet 5 · tool · anthropic.com · paid
Anthropic model priced at $2/M input and $10/M output tokens.
Also in: Claude Haiku 5.5 in Devin: beats Sonnet 5 at about 1/8 the cost (Melvin Vivas on X · notes), Claude Sonnet 5 Pricing Made Permanent ($2/$10 per M Tokens) (Melvin Vivas on X · notes) - Claude Opus 4.7 · tool · anthropic.com · paid
Anthropic's top Opus model, aimed at long-running agentic tasks that need precise instruction following.
Also in: Shipping Features Fast in Cursor with Multiple AI Models (Melvin Vivas on X · notes), Composer 2.5 as the Default Coding Model in Cursor (Melvin Vivas on X · notes), Which AI coding model to use for which task (Melvin Vivas on X · notes), Claude Opus 4.7 Fast Mode in Cursor: Speed vs. Cost Trade-off (Melvin Vivas on X · notes) and 3 more - Grok 3 Mini · tool · docs.x.ai · paid
xAI small model that beat o1 Preview's score for about 1/100 of the price. - Humanity's Last Exam · dataset · lastexam.ai · free
Hard benchmark of expert-level questions used to measure frontier model capability. - AA-Omniscience (Artificial Analysis) · dataset · artificialanalysis.ai · free
Artificial Analysis benchmark that measures knowledge and hallucination rates. - Claude Code · tool · code.claude.com · paid · recommended by both Bashiri Smith & Melvin Vivas
Build agents and pipelines from the terminal; the guide's main agentic coding tool.
Also in: First impressions: Claude Code in the cloud is good and fast (Melvin Vivas on X · notes), Why Claude Code feels faster: gzipped turns and instant cloud sessions (Melvin Vivas on X · notes), Watch What Fills Your Coding Agent's Context and When to Clear (Melvin Vivas on X · notes), Claude Code's Right Sidebar and Context Breakdown (Melvin Vivas on X · notes) and 124 more - OpenAI Codex · tool · openai.com · paid · recommended by both Bashiri Smith & Melvin Vivas
OpenAI's coding agent. In the diagram it writes code, fixes review findings and drives the build loop. The creator also used it to make this video.
Also in: Developers moving from ChatGPT/Codex to Claude (Melvin Vivas on X · notes), Creator Opinion: Codex Is Still the Best at Computer Use (Melvin Vivas on X · notes), OpenAI Codex Ambassador applications are open (Melvin Vivas on X · notes), GPT-Live-1.1: Multilingual Voice Update for ChatGPT and Codex (Melvin Vivas on X · notes) and 264 more - ThePrimeagen · person · x.com · free
Developer and streamer whose tweet about AI getting more expensive inspired the video.
Also in: Prime Agent: A Self-Improving RLM Coding Harness with Code-Based Tool Calls (Melvin Vivas on X · notes) - Theo (t3.gg) · person · x.com · free
Developer and YouTuber who made the video Melvin Vivas reposted. - AI isn't going to keep improving (Theo video) · video · youtube.com · free
Theo's video from about two years earlier arguing that LLMs had plateaued, a take he now calls wrong. - Lakebed · tool · lakebed.dev · check price
Theo's own cloud platform, built on top of Railway.
Try this
- Compare models by cost per task and cost per intelligence level, not just price per token.
- Use Artificial Analysis cost-vs-intelligence charts and the Pareto frontier to find the cheapest model that is good enough for your task.
- Pick models by workload: lowest input-token price for jobs that ingest a lot of text, and token efficiency, caching and reliability for agentic coding.
- Before defaulting to a frontier model, try cheap capable models like GLM 5.3 Flash or DeepSeek V4 Flash and give them good tools such as search.
- Set up agent-managed infrastructure: let a coding agent (Claude Code or Codex) create production, staging and per-PR environments through a platform MCP server such as Railway's.
- Build a simple cost tracker that logs tokens and cost per task across models, to find your own price-performance sweet spot.
More in LLM Fundamentals
- Muse Spark Open: Meta's 816B MoE Open-Weight Model Specs
- New Gemma Developer Documentation Site
- Grok Voice 3.0: xAI's Speech-to-Speech Model
- Liquid AI's LFM2.5-24B-A2B: a local MoE model for agent tasks
- Muse Wonder vs Opus 5.5: Same Prompt, Very Different API Cost
- AI Engineer Vocab Quiz: 11 Acronyms to Know for Interviews