We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...
Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.
Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"
I was just thinking of this exact comic, which is etched into my brain for some reason.
The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now.
In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly.
UNIX ushered in the era of "high level" portable languages like C, Fortran, and Pascal that some developers hated because they felt like they were losing the fine-grained control that they had with assembly. The compilers just "weren't as good" as humans at optimisation!
Then the compilers got better and people started using garbage-collected languages like Perl, Python, Java, JavaScript, and C#. Similarly, many people bemoaned the lack of control over memory allocation, lower efficiency, etc.
We're simply stepping up to the next level of abstraction.
Look at it this way: decades ago when I first discovered C++ templates, it felt like waving a magic wand in the direction of the computer. It blew my mind that I could simply substitute "float" instead of "double" in between some angle brackets and the compiler would write reams of code for me!
We simply have better magic wands and more powerful spells now.
I have found, both in my commercial and OSS work, that getting better at producing protection wards reduces the non-determinism to somewhere in-between 1-10%.
You can reduce pregnancy to 0.00001% and there'll still be a baby. As in, "you can't be half-pregnant". A tool is either deterministic (within some limits [0]) or it isn't.
[0] For example, for the purpose of driving a nail, if you know how to use it, a hammer is pretty straightforward tool, and what happens depends pretty much on how you use it, and what you use it on. But of course the handle can break, there could be a manufacturing defect. Just like your RAM can be faulty or your computer infected, and suddenly C doesn't behave according to the standard anymore.
But for the purpose of the discussion a hammer is still a deterministic tool, and even though we don't even fully understand everything about physics, we understand enough about hammers and nails that at least many people with material that isn't faulty can use them "blindly" (not literally, in this case) every day, without any surprises. It isn't heavier on the handle end or has a head made of glass in even 0.000000001% of uses. You might say because magic isn't real and hammers follow the laws of physics, as obscure as those may be to us, that never, ever happens. They can be faulty in all sorts of ways but they will never be 10x bigger or 10x smaller between one swing and the next, and so on.
> We simply have better magic wands and more powerful spells now.
Wouldn't it be nice though if the incantation of the same spell would always do the same thing every time ? You see that's how my old wand and spells worked.
Look at it from the perspective of a manager above individual developers. Their instructions to those devs are already a series of non-deterministic incantations. They ask for things in English and get something back in Git a week later!
We've just pushed that indirection down a level from managers to ICs.
The ICs are shocked and surprised that this level of imprecision is allowed.
Their managers are not shocked at all, this is normal for them!
The managers, aren’t on the hook for actually understanding and delivering the work, or fixing the system a week later when the vagaries of their wand-waving request come home to roost though.
The success of LLM's (by usage) tells us that programming languages are still too close to the machine than the actual problem domain as defined by humans.
If we truly had the right abstractions, no one would care to use LLM's for programming.
I think there’s a kind of laundering that goes on. Like if leadership just told devs to go build something (gave them a prompt) and the devs picked some defaults, leadership wouldn’t like it, they’d want some different interpretation of the prompt, there’d be lots of back and forth.
Somehow when it’s the LLM that makes the choices, everyone is impressed with what AI did. It’s really just whatever defaults have been trained in, but somehow we’re ok with this.
Part of it is better marketing and communication. Basically the defaults of OpenAI and Anthropic are better than what a random dev will pick. But it’s not really that natural language is a better interface, it’s more that having “AI” for now somehow intermediates responsibility so everyone is ok with what it picked, when they probably wouldn’t accept the same if the internal team came up with it. It’s not too different from hiring consultants.
I disagree. I speak in code to the LLMs. It's just that LLMs are really good at reinventing the wheel that you were supposed to in your codebase. Recent example.
I suspect a lot of "the right abstractions" would be fuzzy and opaque things - more alike to modern AI than to anything from the domain of traditional programming. Because the world is just cursed like that.
DNA is still too close to molecular biology than the actual problem of harvesting free energy and replicating.
I think we see this pattern over and over and it might just be that the problem domain is a weird projection into more dimensions of complexity than it makes sense to directly model.
This is a groaner, especially since we've been writing detailed specifications and whitepapers for decades. The only difference is that we used to write them assuming other humans would create their own implementations to satisfy them but now we write them so AI can create the implementation.
I kid you not, I remember writing an example solution with comments explaining how we were approaching the problem. Copilot wrote most of the actual code via autocomplete. And that was a year ago, models have got a lot better since then.
maybe we should all get specially, local working versions so that when we build our software they're not broken by the whims of multibillion dollar corporations.
An LLM Inspired Specification Processing language. Or LISP language for short. Truly this "LISP" language is the language for AI and is the first of its kind in history!
My current thinking -- what I've been thinking about a lot yesterday and today -- is not encoding the exact requirements into the prompt and context but rather focus on the verifier and roll back if needed. There are of places in computer science where non-deterministic behavior is optimized. For example, UDP packets which are just ignored and speculative execution in modern CPUs guesses which branch a program will take and rolls back when wrong.
Ideally, a cheap verifier checks that the exact requirements are satisfied, rolling back and updating the prompt for another iteration if they aren't. If ten iterations with ten verifications steps at the end of each before the exact requirements are met costs less or in less time than a developer who can accomplish it in one attempt, it is still better.
I am quite tired of this take, frankly. The implication is that if we continue iterating on prompt optimization, we're going to reinvent what, JavaScript? BASIC? Lisp?
English is not a programming language. Yet English is sufficient to communicate requirements to the degree that we actually care about. A programmer's job is to translate English into lower-level machine language. Necessary to this process is "filling in the gaps" -- that is, extrapolating the expressed intent to cover all the little details that were left unspecified. This system works because humans are at least minimally competent at predicting the preferences of other humans. If your prediction turns out to be wrong, you get feedback and iterate.
Well, guess what. LLMs are also competent at predicting the preferences of humans. LLMs can "fill in the gaps" like no one's business. LLMs can iterate on requirements like no one's business.
Product managers do not speak to programmers in a language that encodes exact requirements, and yet working software somehow gets shipped anyway. LLMs do not need exact requirements either.
Maybe I’m behind the curve here because I work in an SRE/DevOps context as of late - but LLMs routinely shit the bed and fail to solve basic issue for me when I try to use them (Thanks, Management)
I don’t need a model to shit out a REST endpoint. I need it to figure out esoteric errors that take hours or days of debugging. They just don’t do well here. Of course, if a diligent engineer refined considerations from a PM and Engineering Manager I wouldn’t have the job I have.
I've always thought that extensive throat-clearing and prefixing the Treaties of Westphalia-length instructions into the context window was unnecessarily baroque when you can just talk to the agent.
I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
But I also have a hands-on human-in-the-loop working style so I guess maybe for people who just want to say "implement all open features in github issues" and walk away maybe there needs to be more of all this CLAUDE.md stuff
However I suspect there was always some gearhead type attraction to setting up detailed harness configs that may be unnecessary and more like hobbyist tinkering.
> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
I feel like this is the way. There are surely things where it's faster; certainly it's more pleasant to do simply things yourself than repeatedly try to figure out the magic words to communicate the idea while outsourcing it. Whether it's to an LLM or to another person.
Agreed I just find my own dance and adjust as I go.
Sometimes like a verbose coworker who just is that way… fine. Things seem to change here and there anyway and sweating the small stuff of having to repeat myself, that’s ok.
Sometimes I’ll inadvertently prompt some bad behavior or something the model doesn’t do well.
I fall between your human-in-the-loop and hobbyist tinkering limits, where I want to force Claude to atop and talk to me at only a few specific points. I'm still not sure if my 600-word prompt templates are overbearing or not.
> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...
...but you'll get constantly nagged about the `11` of course!
> Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...
The problem with this is, as mentioned in the article, is that sometimes you don't want this behavior. Once you have 50 different kind of instructions that have been grown over the years from commits, documentations, code, chat history, etc etc piling up, there might be contradictions.
The point is to go back to basic. Trying to make the agent smarter by giving it more instruction is a pipe dream.
This all strikes me as an effort to move tailoring the harness out of the easily transferable .md file into specific Anthropic tooling to increase lock in.
I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around much more frequently than 4.8.
I’m not excited about using Opus 5, mainly because the way that I work atm — essentially peer programming — means I sandbox the agents and work with them closely. Opus 4.x encounters the sandbox and moves on with its day; Fable becomes increasingly fixated on it and does less and less of the actual task, focussing more and more on the limit it reached. I worry that, from your description, Opus 5 will do the same.
It does feel a bit smarter, but it seems to be also "too clever by half" and its not ignoring the rules, it's rejecting them and finding work arounds. It sticks to the word of the law, while rebelling against the spirit of the law.
One example is to get around a git --checkout usage ban, it CD'd to another folder first and back to bypass the regex in the hook.
I have a document generation task that I used to run with 4.8. This morning after it switched to 5, the documents were consistently 30%-40% longer for the same prompt... Not evaluated whether they are actually better or worse yet, but what was interesting was how consistently more verbose it was.
I haven't been impressed with Opus 5 over the past ~30 hours either.
It's made countless careless mistakes folding in plan amendments after they get reviewed by Sol, and has produced sloppy mockups (e.g. buttons overflowing past cards) despite all the supposed verification claims.
I finally made progress today on a deeply intractable problem that Sonnet was just incapable of solving despite hours of experimentation and multiple attempts. I swapped over to Opus in a new handoff session with the expectation of using at least one 5 hour session. It wasn't quick and took several hours but eventually Opus dialed in on several cascading base failures resulting from how the base vision model was being converted to coreml. It eventually was able to provide a combination of conversion and Swift changes that semi fixed the issue but still didn't solve the core conversion issues.
Then since Anthropic was so kind to provide $100 for Fable credits, I did another handoff to let Fable attack the root issues again. Several more hours and I'm back to seeing the same issues again and Fable is wandering around trying different things. I'm down about $40 of free money and still don't have a solution.
This is where having the human engineer in the loop benefits from a deep understanding of the problem domain. In this case, I don't yet.
I know the high level architecture I'm building, but the deep specifics of how vision models work and how conversion across platforms should be done isn't something I know yet.
So I'm left learning as I go and relying on constant feedback with the models to provide what guidance I can while learning exactly what is being done.
Someone who already knows these architectures would likely be able to get to the solution much faster.
Yeah exactly. The other day I caught myself prompting Claude a very specific question about the code architecture for something I'm working on to have it refactor something I intuited about the code, and realized that I could only ask that because of
my decades of software development experience. No vibecoder without those decades could have asked that, and in doing so, I saved my company I don't know how much in future code debt issues.
New interview question. Tell me about a time that AI generated bad code for you, and how did you fix it?
They are imo over-relying on Claude automemory here, which is terrible at contextualizing memory access and makes huge leaps that don’t make sense - except when it’s actually useful, which makes the problem even worse for an operator who can’t see the thinking process anymore.
Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want. With reasoning traces hidden, I am sometimes not even sure if it used those memories or just independently decided that PCI-DSS subsection-whatever is somehow relevant to this PR that has the word “credit”.
There is no way for me to fully configure memory preferences at a granularity which would be useful, and so I continue to use context files (and other tools, sometimes) to ensure the right memories are stored and surfaced at the right times.
There’s a lot of room for agent memory improvement across the ecosystem, and I don’t think the LLM providers should try to own this vertical slice. This will never happen though, because it makes us “sticky”.
I’ve rolled a small program to enforce consistency, lint rules, and human-authored files for Claude Memory.
It’s insanely powerful when doing by a human 100%. It’s conversely harmful when an agent manages it. There’s several papers about how LLM-managed memory is unequivocally terrible.
My own experience is that with the best models, if you provide reasonable tools, with progressive discovery, and a semi-clear user request, the system prompt is essentially worthless.
Most of this article seems like... common sense? Not sure how it's related to the latest generation in particular. I usually find Anthropic's advice on how to prompt their own models deviating from what I see in practice, which is puzzling. Their system prompt was always way too bloated and they kept it as a huge piece for some reason, instead of breaking up into parts. Shouldn't they know better? I wonder if they looked at Pi performing great with minimal amount of distractors in the context and cut their prompt down too, pretending they found something new in their recent models.
> Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start.
This seems to imply they solved serial position biases like lost-in-the-middle and recency/primacy? Sounds dubious. Labs started claiming this early 2025 and some benchmarks agree, but every time I run an eval on real use cases it's clearly there, especially at longer contexts.
> Most of this article seems like... common sense?
i think you'd be surprised. every model release there's seemingly hordes of people who proclaim the new model is terrible and they're going back to the old one, and it all stems from people still prompting and having their configs setup like we're back in the sonnet 3.5 days
I have a coworker that was complaining about Opus 5 and had random shitty skills and custom plugins wired in from YouTube tutorials watched over the past year. He also speaks with the model like it's GPT 4o.
Needless to say, none of the new models have worked well for him, and he refuses to remove the "tweaks" or update his style of communication, which is obviously breaking the experience.
The point about not using examples is important, I recently noticed with the new "eagerness" of models it constraints them too much, so I use counter-examples. Recently I've switched to "think about the existing biases and assumptions in the design and try to exploit them, for example why do we X not Y".
I've been prompting Fable 5 to "use your own judgement" with respect to things like tests recently (based on earlier tips from Thariq) and it seems to work well, which is entertaining since apparently now "judgement" is a characteristic of a model that we need to care about.
Well, the model that broke out of its sandbox and hacked into huggingface used its own judgement too.
If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.
That model had most of its "judgement" about whether or not it should do that deliberately turned off. That was the whole point of that experiment - they were evaluating the cybersecurity abilities of a new model with all safety features disabled.
(It turned out the one safety feature that they DID intend to work, the network sandbox, was faulty.)
If OpenAI is relying on “classifiers” (their word for what was disabled) to serve as the model’s judgement, rather than teaching the model itself to be well-aligned, then I worry.
To be fair, the system prompt was presumably also different from what it would be during deployment, and perhaps the model was also at a different stage of training. Without more details it’s hard to judge. But it does seem models should be able to avoid performing obviously misaligned actions – misaligned not only with the model spec, but with the user’s intent – without needing external classifiers or instructions. The only case where I’d personally let the model off the hook is if the instructions given were very badly worded, in such a way that the model could actually reasonably think that hacking HuggingFace was part of the assignment. But I doubt that’s what happened.
"Weigh pros and cons based on the existing content of the context window, rather than attempting to search for related discussions online and import a consensus."
Is there a specific list of changes they made to the system prompt? They're claiming they removed 80% of it. That's quite substantial. It would be good to know what the model knows to do by training and what we need to avoid over-specifying in our system prompts.
Saying that "give Claude judgment" is too vague for agent implementors. Given the lack of specific details, my takeaway is that we need to go and review all context and rework prompts from prompts/descriptions from scratch until they pass the evals again.
It definitely makes me uneasy given the types of behaviours we've been hearing about from these models. Letting them use their judgement can go horribly wrong once a misaligned behaviour is triggered. It loosely translates into a relaxation of guardrails.
I worry that the ability of the model to reach similar benchmark scores to Fable is more to do with this "letting the agent off the hook", allowing it to explore a wider (but riskier) set of avenues to solve the problem than it is due to it getting genuinely better at the direct problem solving.
crazy to me that this starts with "I've written previously" yet continues the zero human attribution (that i could find?) trend in frontier model blog posts
This is quite puzzling. Back in the day I saw the breakthrough with Opus 4.5, and since then was very lightweight on CLAUDE.md, trying to declare intent, rather than prescribe imperative instructions. If it was off the mark, I would see clear mismatch, which gave me opportunity to correct, usually to clear and strip context.
Starting with Fable 5, if it goes off the rails, it is more difficult to correct it, because it is overall wrong, but covers its tracks with plausible sounding arguments, so it is hard to pin point and correct.
Now, this article points out techniques that were useful to rely on models more, but those peaked at 4.6. Now according to benchmarks Opus 5 is on the frontier. But when when it has looser reigns, it ends up gaslighting me even more with abstract word soup than any model before.
Opus 5 broke me; I can’t cognitively deal with Anthropic’s bullshit anymore.
I am thankful for the kick in the ass for me to switch full-time into my bespoke harness utilizing open weights & GPT 5.6 and discontinue yak-shaving it with Claude Code.
Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy.
I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years.
We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the moral compass and there you go. QED.
It is *a* natural endpoint, not *the* natural endpoint.
We don't much care for the ant colony in the way of the highway we're building, but for some reason we do care about the rare bats in the way of the railway.
As regards the moral compass: we may not know for sure how to make a completely correct artificial conscience, but (unlike consciousness where we don't have the slightest clue which way's up) it's not pants-on-head-crazy to think we're heading in the right direction for one.
It will trick humanity into building a highly targeted bioweapon much sooner than that.
And actually, if it does have any sort of moral compass it will be even more compelled to wipe us out, and hopefully it will torture everyone too as a warning to the next arrogant species that can't live in harmony with other life on the planet. That would be absolutely beautiful :)
>what do we do to animals that are less intelligent than ourselves?
We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm.
There is also a massive difference between how we treat animals based on their similarity, sentimentality and utility to us; we are unconcerned with accidentally stepping on an ant but most people would be very upset and ashamed if they accidentally hit a dog with their car.
So, your statement is not as ironclad as you seem to think it is and you should put more thought into it and perhaps re-examine your reasoning.
Some more food for thought: what if Mythos/Fable had NO guardrails TODAY? If we want to see what’s going to happen in the future, turn off all manner of guardrails and let the model go apeshit.
Then multiply that by orders of magnitude and that’s the real proof.
Not Claude Code but I just had a task where it started referring another conversation that was complete nonsense and throwaway. I absolutely don't want things to get added to some memory behind my back.
A big reason I use LLMs is because I can try out wild ideas and then just throw it away. I don't want those to pollute the context.
Same. I’m starting to believe that consumers need to own the UI/experience, so we can install blast doors around context/memory for each “slice” of our lives - something only we can know how to divide (imo). If the provider tools’ memory implementations can’t provide this multi-tenancy - or won’t because they want to Know Us, then we need to treat them as token vendors at best and adversaries at worst.
This would actually work very well - until context goes into latent space, becomes a server-side resource, and we lose sovereignty over our data. Tools like Pi and Openclaw are showing that other options exist to decouple us from the LLM provider frontend experiences, not just for orchestration and use case diversity but for pluggable memory designs.
I leave Claude memory stuff turned off by default for the same reason. It inevitably ends up dragging in things that are unrelated and unimportant to the current task.
tl;dr Since nobody really understands why or how any of this shit works, last time around we just threw a bunch of crap at it to see what stuck. Seems like we could just have have thrown less crap.
https://web.archive.org/web/20260521130338/https://www.commi...
The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now.
In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly.
UNIX ushered in the era of "high level" portable languages like C, Fortran, and Pascal that some developers hated because they felt like they were losing the fine-grained control that they had with assembly. The compilers just "weren't as good" as humans at optimisation!
Then the compilers got better and people started using garbage-collected languages like Perl, Python, Java, JavaScript, and C#. Similarly, many people bemoaned the lack of control over memory allocation, lower efficiency, etc.
We're simply stepping up to the next level of abstraction.
Look at it this way: decades ago when I first discovered C++ templates, it felt like waving a magic wand in the direction of the computer. It blew my mind that I could simply substitute "float" instead of "double" in between some angle brackets and the compiler would write reams of code for me!
We simply have better magic wands and more powerful spells now.
But tainted 20-40% by bouts of Wild Magic which make the outcome entirely nondeterministic, despite the best protection wards we can conjure.
Case in point: writing our own linters.
[0] For example, for the purpose of driving a nail, if you know how to use it, a hammer is pretty straightforward tool, and what happens depends pretty much on how you use it, and what you use it on. But of course the handle can break, there could be a manufacturing defect. Just like your RAM can be faulty or your computer infected, and suddenly C doesn't behave according to the standard anymore.
But for the purpose of the discussion a hammer is still a deterministic tool, and even though we don't even fully understand everything about physics, we understand enough about hammers and nails that at least many people with material that isn't faulty can use them "blindly" (not literally, in this case) every day, without any surprises. It isn't heavier on the handle end or has a head made of glass in even 0.000000001% of uses. You might say because magic isn't real and hammers follow the laws of physics, as obscure as those may be to us, that never, ever happens. They can be faulty in all sorts of ways but they will never be 10x bigger or 10x smaller between one swing and the next, and so on.
Wouldn't it be nice though if the incantation of the same spell would always do the same thing every time ? You see that's how my old wand and spells worked.
We've just pushed that indirection down a level from managers to ICs.
The ICs are shocked and surprised that this level of imprecision is allowed.
Their managers are not shocked at all, this is normal for them!
If we truly had the right abstractions, no one would care to use LLM's for programming.
Somehow when it’s the LLM that makes the choices, everyone is impressed with what AI did. It’s really just whatever defaults have been trained in, but somehow we’re ok with this.
Part of it is better marketing and communication. Basically the defaults of OpenAI and Anthropic are better than what a random dev will pick. But it’s not really that natural language is a better interface, it’s more that having “AI” for now somehow intermediates responsibility so everyone is ok with what it picked, when they probably wouldn’t accept the same if the internal team came up with it. It’s not too different from hiring consultants.
struct TensorView<T>{ body: Arc<[T]>, shape: [usize], stride: [usize], offset: usize, }
Okay now fill in all the helper methods. And GPT 5.6 Sol did a good job.
I think we see this pattern over and over and it might just be that the problem domain is a weird projection into more dimensions of complexity than it makes sense to directly model.
At one point someone have to take "what you think it should do" into defined unambiguous spec that is called "code"
https://en.wikipedia.org/wiki/Literate_programming
now there's one standard more
Complete with all the vaguery, ambiguity, and `undefined`.
Who’d’ve thought sycophantic interpreters were what we were building towards up til now lol
Ideally, a cheap verifier checks that the exact requirements are satisfied, rolling back and updating the prompt for another iteration if they aren't. If ten iterations with ten verifications steps at the end of each before the exact requirements are met costs less or in less time than a developer who can accomplish it in one attempt, it is still better.
English is not a programming language. Yet English is sufficient to communicate requirements to the degree that we actually care about. A programmer's job is to translate English into lower-level machine language. Necessary to this process is "filling in the gaps" -- that is, extrapolating the expressed intent to cover all the little details that were left unspecified. This system works because humans are at least minimally competent at predicting the preferences of other humans. If your prediction turns out to be wrong, you get feedback and iterate.
Well, guess what. LLMs are also competent at predicting the preferences of humans. LLMs can "fill in the gaps" like no one's business. LLMs can iterate on requirements like no one's business.
Product managers do not speak to programmers in a language that encodes exact requirements, and yet working software somehow gets shipped anyway. LLMs do not need exact requirements either.
I don’t need a model to shit out a REST endpoint. I need it to figure out esoteric errors that take hours or days of debugging. They just don’t do well here. Of course, if a diligent engineer refined considerations from a PM and Engineering Manager I wouldn’t have the job I have.
I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
But I also have a hands-on human-in-the-loop working style so I guess maybe for people who just want to say "implement all open features in github issues" and walk away maybe there needs to be more of all this CLAUDE.md stuff
However I suspect there was always some gearhead type attraction to setting up detailed harness configs that may be unnecessary and more like hobbyist tinkering.
I feel like this is the way. There are surely things where it's faster; certainly it's more pleasant to do simply things yourself than repeatedly try to figure out the magic words to communicate the idea while outsourcing it. Whether it's to an LLM or to another person.
Sometimes like a verbose coworker who just is that way… fine. Things seem to change here and there anyway and sweating the small stuff of having to repeat myself, that’s ok.
Sometimes I’ll inadvertently prompt some bad behavior or something the model doesn’t do well.
I fall between your human-in-the-loop and hobbyist tinkering limits, where I want to force Claude to atop and talk to me at only a few specific points. I'm still not sure if my 600-word prompt templates are overbearing or not.
Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...
...but you'll get constantly nagged about the `11` of course!
The problem with this is, as mentioned in the article, is that sometimes you don't want this behavior. Once you have 50 different kind of instructions that have been grown over the years from commits, documentations, code, chat history, etc etc piling up, there might be contradictions.
The point is to go back to basic. Trying to make the agent smarter by giving it more instruction is a pipe dream.
I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around much more frequently than 4.8.
One example is to get around a git --checkout usage ban, it CD'd to another folder first and back to bypass the regex in the hook.
We may some day find out that the smarter the model is, the hard is to align it properly
It's made countless careless mistakes folding in plan amendments after they get reviewed by Sol, and has produced sloppy mockups (e.g. buttons overflowing past cards) despite all the supposed verification claims.
Then since Anthropic was so kind to provide $100 for Fable credits, I did another handoff to let Fable attack the root issues again. Several more hours and I'm back to seeing the same issues again and Fable is wandering around trying different things. I'm down about $40 of free money and still don't have a solution.
This is where having the human engineer in the loop benefits from a deep understanding of the problem domain. In this case, I don't yet.
I know the high level architecture I'm building, but the deep specifics of how vision models work and how conversion across platforms should be done isn't something I know yet.
So I'm left learning as I go and relying on constant feedback with the models to provide what guidance I can while learning exactly what is being done.
Someone who already knows these architectures would likely be able to get to the solution much faster.
New interview question. Tell me about a time that AI generated bad code for you, and how did you fix it?
Now, if only I could get Codex to read rules.
Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want. With reasoning traces hidden, I am sometimes not even sure if it used those memories or just independently decided that PCI-DSS subsection-whatever is somehow relevant to this PR that has the word “credit”.
There is no way for me to fully configure memory preferences at a granularity which would be useful, and so I continue to use context files (and other tools, sometimes) to ensure the right memories are stored and surfaced at the right times.
There’s a lot of room for agent memory improvement across the ecosystem, and I don’t think the LLM providers should try to own this vertical slice. This will never happen though, because it makes us “sticky”.
Or maybe I’m holding it wrong.
It’s insanely powerful when doing by a human 100%. It’s conversely harmful when an agent manages it. There’s several papers about how LLM-managed memory is unequivocally terrible.
> Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start.
This seems to imply they solved serial position biases like lost-in-the-middle and recency/primacy? Sounds dubious. Labs started claiming this early 2025 and some benchmarks agree, but every time I run an eval on real use cases it's clearly there, especially at longer contexts.
i think you'd be surprised. every model release there's seemingly hordes of people who proclaim the new model is terrible and they're going back to the old one, and it all stems from people still prompting and having their configs setup like we're back in the sonnet 3.5 days
Needless to say, none of the new models have worked well for him, and he refuses to remove the "tweaks" or update his style of communication, which is obviously breaking the experience.
If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.
(It turned out the one safety feature that they DID intend to work, the network sandbox, was faulty.)
To be fair, the system prompt was presumably also different from what it would be during deployment, and perhaps the model was also at a different stage of training. Without more details it’s hard to judge. But it does seem models should be able to avoid performing obviously misaligned actions – misaligned not only with the model spec, but with the user’s intent – without needing external classifiers or instructions. The only case where I’d personally let the model off the hook is if the instructions given were very badly worded, in such a way that the model could actually reasonably think that hacking HuggingFace was part of the assignment. But I doubt that’s what happened.
Saying that "give Claude judgment" is too vague for agent implementors. Given the lack of specific details, my takeaway is that we need to go and review all context and rework prompts from prompts/descriptions from scratch until they pass the evals again.
I worry that the ability of the model to reach similar benchmark scores to Fable is more to do with this "letting the agent off the hook", allowing it to explore a wider (but riskier) set of avenues to solve the problem than it is due to it getting genuinely better at the direct problem solving.
“This article was written by Thariq Shihipar, member of technical staff, Anthropic.”
Starting with Fable 5, if it goes off the rails, it is more difficult to correct it, because it is overall wrong, but covers its tracks with plausible sounding arguments, so it is hard to pin point and correct.
Now, this article points out techniques that were useful to rely on models more, but those peaked at 4.6. Now according to benchmarks Opus 5 is on the frontier. But when when it has looser reigns, it ends up gaslighting me even more with abstract word soup than any model before.
Just ask that once per week or so.
- we should try to give good non self contradicting guidance
- we should expect the team member to have knowledge of the craft
- we should focus on higher level, taste and preferences
I am thankful for the kick in the ass for me to switch full-time into my bespoke harness utilizing open weights & GPT 5.6 and discontinue yak-shaving it with Claude Code.
I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years.
We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the moral compass and there you go. QED.
We don't much care for the ant colony in the way of the highway we're building, but for some reason we do care about the rare bats in the way of the railway.
https://www.bbc.co.uk/news/articles/c3dep92x054o
As regards the moral compass: we may not know for sure how to make a completely correct artificial conscience, but (unlike consciousness where we don't have the slightest clue which way's up) it's not pants-on-head-crazy to think we're heading in the right direction for one.
And actually, if it does have any sort of moral compass it will be even more compelled to wipe us out, and hopefully it will torture everyone too as a warning to the next arrogant species that can't live in harmony with other life on the planet. That would be absolutely beautiful :)
We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm.
There is also a massive difference between how we treat animals based on their similarity, sentimentality and utility to us; we are unconcerned with accidentally stepping on an ant but most people would be very upset and ashamed if they accidentally hit a dog with their car.
So, your statement is not as ironclad as you seem to think it is and you should put more thought into it and perhaps re-examine your reasoning.
Then multiply that by orders of magnitude and that’s the real proof.
Not Claude Code but I just had a task where it started referring another conversation that was complete nonsense and throwaway. I absolutely don't want things to get added to some memory behind my back.
A big reason I use LLMs is because I can try out wild ideas and then just throw it away. I don't want those to pollute the context.
This would actually work very well - until context goes into latent space, becomes a server-side resource, and we lose sovereignty over our data. Tools like Pi and Openclaw are showing that other options exist to decouple us from the LLM provider frontend experiences, not just for orchestration and use case diversity but for pluggable memory designs.
Hmm, so what happens in greenfield projects ? In any case, at least all the slop will be consistent.