I just had Claude try to process an RSS feed and it was about to ZALGΌ IS TOƝȳ THË PO NY itself and I pointed that out and it immediately said "Wordpress has a json interface, I'll use that".
If you draw a sloppy circle and fill it in, it'll "escape" and try to paint the whole canvas (and back in the day would get my slow computer stuck until I spam "esc").
You have to be able to draw a good circle to use it.
Reviewing my experience using LLMs, to improve results, reduce churn and token usage. Discovering the gap between what they produce and what I'd normally do is a significant source of output cost, regressions and surfacing a bit of why and how to fix it. Notably Claude is remarkably bad at/about this, producing errors even when directed toward modern Web solutions—that cut token use a lot, like toward 90% occasionally, which together with the frustrating churn led me to review how I'm working, what is happening and generate this article.
Excellent article, with impeccable analysis, that will fundamentally change how I work with Claude myself. I have already learned to give Claude both a "do" and a "don't" in order to limit unpleasant surprises.
This matches my experience. I've been using OpenRouter with
GPT-4o for an image verification service, and the prompt
engineering choices have a measurable impact on cost.
One thing I found: asking the model to respond in structured
JSON (with a strict schema) vs free-form text cuts token output
by ~40% on average. The model stops "explaining itself" and just
gives you the answer.
Also noticed that including a reference image in vision calls
roughly doubles the input cost but improves accuracy enough that
you save on retries. Net cost ended up lower for my use case.
Curious if you've measured the difference between asking for
"concise" output vs actually constraining the response format.
You need to know the shape of the solution ...
If you draw a sloppy circle and fill it in, it'll "escape" and try to paint the whole canvas (and back in the day would get my slow computer stuck until I spam "esc").
You have to be able to draw a good circle to use it.
https://stackoverflow.com/questions/1732348/regex-match-open...
And it feels like claude code has gotten more verbose with the multiline comments lately
One thing I found: asking the model to respond in structured JSON (with a strict schema) vs free-form text cuts token output by ~40% on average. The model stops "explaining itself" and just gives you the answer.
Also noticed that including a reference image in vision calls roughly doubles the input cost but improves accuracy enough that you save on retries. Net cost ended up lower for my use case.
Curious if you've measured the difference between asking for "concise" output vs actually constraining the response format.