Find the Cheapest Model That Still Ships Your Side Project
Judge a weekend coding agent by the cheapest model that clears your minimum task threshold, then re-baseline cost after every model change.
You open the agent on a weekend, paste a small feature, and watch it use a frontier model. The question is whether the cheapest model that still ships is good enough.
Cognition launched SWE-2 on September 10 in Devin Desktop and CLI. SWE-2 was built on Kimi K3, a 2.8 trillion-parameter mixture-of-experts model from Moonshot AI.
On FrontierCode 1.1 Main, a coding benchmark, SWE-2 scored 50.0%, 0.9 percentage points below Fable 5.1's 50.9%, while the model was reported to run 64% cheaper. On Terminal-Bench 4, SWE-2 scored 27.3%, compared with 55.8% for Fable 5.1 and 57.9% for GPT-6 Astra.
The cheapest model is the one that clears your threshold
Most side projects stay side projects, and that is fine. For a weekend side project, the best coding agent is the one that clears the minimum capability threshold for the task you actually need to finish. The threshold is a practical test: the smallest set of edits, tests, and explanations that lets you deploy something you would not delete the next morning.
The near-tie on one benchmark is not a decision for your project. Use it to decide which tasks are worth testing on the cheaper model. Start with a task you have already shipped once, because it gives you a clean, repeatable before-and-after comparison.
Your task decides. A small API endpoint, a failing test, and a README note may clear the threshold on a cheaper model, while a multi-step terminal workflow may stall the same model. Choose the cheapest model or pairing that still clears the threshold, not the one that wins the benchmark. Track the model's spend separately from your subscription, so you can see which part of the cost changes.
The real cost includes your editing time
Model spend is only part of the bill. SWE-2 medium made its first real code edit after a median of 18 steps, compared with 48 steps for SWE-1.7. On FrontierCode 1.1 Main, SWE-2 medium outscored SWE-1.7 while using 58% fewer turns and incurring 81% lower average cost. For a weekend builder, fewer turns can matter more than a small benchmark gap, because your attention is the scarce resource.
Price alone is not proof of capability. A model can be cheap and still produce edits that need more cleanup than the expensive one. If the cheaper model saves money but adds review time, you have not cut the total cost. Write down the review time before you switch, so the warning becomes data.
A weekend agent needs a maintained cost baseline
- Define the smallest shippable task. You are done when the test passes and the endpoint returns the expected JSON.
- Measure cost per shipped task. Done means a number: model spend plus minutes spent editing, for one repeatable task.
- Re-baseline after each model or pairing change. The result is a note: old model, new model, same task, lower or higher total cost.
Keep a small record of the last task you shipped, the model used, the number of edits you accepted, and the time you spent fixing what it missed. When a new model appears, run the same task, not a new one. Switch when the new model clears the threshold with less total cost. Keep the old setup and note why when it does not. For several task types, keep one baseline per type instead of one average.
Devin Pro costs $20 per month and includes SWE-2 usage at no additional charge through October 10, 2026. Bundled pricing can make a model look like an easy upgrade, but the baseline still matters. When the bundled model is not the right fit for your task, the monthly fee is not the only cost you are paying.
Keep the note short enough that you will actually update it when it matters.