Nolan Lawson benchmarks 12 JavaScript frameworks with Opus 5.5: Angular and SvelteKit cost most, and framework MCPs often go unused

Frontends are now being rewritten because one framework is supposedly "better for agents". Nolan Lawson, the web performance engineer who has spent years benchmarking browsers and frameworks, asked the obvious question, "Did anyone think to ask the agents?", and measured it. His post "The AI cost of JavaScript frameworks" (October 28) had Claude Code with Opus 5.5 at high effort build the same app in 12 frameworks, five times each, and compared cost, time, tokens, code size and Lighthouse scores. The harness is public in nolanlawson/agent-framework-benchmark.

The app is a small but non-trivial Pokémon browser on top of PokéAPI, with lists, types, moves, search and filters. Each agent ran in a sandbox, was told to use the latest version of its framework and had the framework's official MCP server or skills installed where one existed. It got a conformance test suite built on the accessibility tree rather than the DOM, so that a web-components app using shadow DOM could pass too. It was not told about Lighthouse, not told to optimize, and pre-generating static HTML for every Pokémon was forbidden and checked.

Cost per run

Medians of five runs. Dollar figures are API prices as reported by Claude Code; Lawson himself ran everything on a $100/month Max plan.

The spread between runs is wide: the cheapest React Router run ($2.82) beat the most expensive Hono+htmx run ($3.19). But three frameworks were reliably more expensive, and the reasons are more interesting than the ranking.

• SvelteKit: in every run, the agent read the SvelteKit v3 migration guide. Lawson thinks v3 (the benchmark used @sveltejs/kit 3.1.0) is too new to be in Opus 5.5's training data, so the agent paid to catch up.
• Gracile+Lit: the agent didn't know the framework well and either cloned the Gracile repository or read its compiled source in node_modules. In two runs it didn't use Lit at all.
• Angular: mostly sheer code size. In two of five runs the agent also built an SSR app that rejected requests from anything but localhost, although the instructions said it would sit behind a reverse proxy. Lawson counted those as spec failures, so Angular's size and performance numbers use three runs and its cost uses all five.

A framework that is newer than the model's training cutoff, or simply less common, costs real tokens every time an agent has to relearn it.

The AI tooling mostly didn't help

The frameworks' own agent tooling, meant to fix exactly that problem, was used unevenly. Nuxt's MCP server was never used. Next.js agents read the bundled skill file in one of five runs and AGENTS.md in two. TanStack Start's skill files, shipped inside node_modules, were found in two runs. Astro's MCP was used in two. At the other end, the Hono and React Router skills were used in all five runs, and so was the MCP in Svelte's Claude Code plugin, though SvelteKit still came out among the most expensive.

Lawson argues the test is fair anyway, since few people are "shouting at their agents in all-caps to use the MCP", and goes further: "I have a strong suspicion, for example, that the current crop of MCPs and skills may be doing more harm than good."

What users get

Lighthouse performance, accessibility and layout-shift scores came out almost saturated: everything scored 97 to 100. The differences show up in shipped JavaScript, from 0 KiB for Astro and 7 KiB for the framework-free version to 115 KiB for React Router, 118 KiB for TanStack Start and 142 KiB for Next.js. Largest Contentful Paint broadly followed bundle size, from 810 ms for Astro, under simulated mobile throttling, to about two seconds for Next.js, with Qwik the notable SPA that scored with the multi-page apps. Left to choose for itself, the framework-free agent built a single-page app all five times.

For maintainability Lawson used lines of code, which with agents also means future token cost. Hono+htmx needed a median of 1,370 lines, Angular 2,009: roughly the same app in about 70% of the code.

What it doesn't show

Lawson lists the limits himself. It is one model, one agent and one effort level. The app is a content-heavy site, "closer to a Wikipedia than a Figma", and Lighthouse is biased against SPAs, so multi-page frameworks were favored from the start; a dashboard or editor might rank them differently. Agents are stochastic, LCP picks different "largest" elements between runs, and five iterations only partly smooth that out. He also hasn't tried runs with and without the MCPs and skills, which is the comparison that would test his suspicion about them. The benchmark is open for anyone to fork, and he has offered to report results from others, including the framework authors.