Be Skeptical of Model Leaderboards: Muse Spark vs Astra

Melvin Vivas · X post · 2026-09-04 · Open on X

Topics: Evaluation (Evals) & Testing, Industry Trends & Job Market · Level: beginner

Summary

The creator tried Muse Spark 1.3 in OpenCode and found it 'no way close to a GPT model'. He doubts an Artificial Analysis chart that ranks Muse Spark 1.3 Max above Astra, though he notes he tested the non-Max version. The lesson: check leaderboard rankings by testing models on your own tasks.

Key points

• Hands-on, Muse Spark 1.3 felt much weaker than GPT models on coding.
• The Artificial Analysis intelligence index reportedly ranked Muse Spark 1.3 Max above GPT-6 Astra, which the creator and the quoted poster doubt.
• Model versions matter: the base model and the 'Max' version can score very differently, so compare the same versions.
• Don't rely on one aggregate index. Test models yourself on real tasks.

Resources mentioned

• Artificial Analysis · website · artificialanalysis.ai · free
  Independent benchmarking site that compares AI models and providers, including text-to-speech, on quality, speed and price.
  Also in: ElevenLabs Eleven v4 & v4 Turbo: Emotive, Multilingual Text-to-Speech Models (Melvin Vivas on X · notes), Pipette: Open-Source Benchmarking for On-Device Models (Melvin Vivas on X · notes), GLM 5.2 Hits 446 tok/s on Fireworks AI (Melvin Vivas on X · notes), GLM-5.2 on Fireworks: Top Open-Weights Model on GDPval-AA (Melvin Vivas on X · notes) and 1 more
• OpenCode · tool · x.com · free
  Open-source terminal coding agent that can be configured with different model providers, including open models.
  Also in: T3Code: a better interface for terminal coding agents (Melvin Vivas on X · notes), OpenCode Go: Open Coding Models (DeepSeek, Qwen, Kimi, GLM) for $10/mo (Melvin Vivas on X · notes), Trying OpenCode Go After Upgrading OpenCode (Melvin Vivas on X · notes), Orchestrator + Subagents in opencode with Open Models (Melvin Vivas on X · notes) and 10 more
• Muse Spark 1.3 · tool · research.meta.ai · free
  Meta's model release, available for free in OpenCode at the time of the post.
  Also in: Try Muse Spark 1.3 for Free in OpenCode (Melvin Vivas on X · notes), Muse Spark 1.3 Is Free to Try in OpenCode (Melvin Vivas on X · notes), Installing Meta's Muse Code CLI (Needs a Paid Plan) (Melvin Vivas on X · notes)

Try this

• Check leaderboard rankings by testing models yourself on your own tasks.