Interesting project. I am working on a similar solution. Eventually you will run into the following with harnesses, so I wonder how these questions work with your project;
1) Can you define a process other than build -> review -> .. etc. And more importantly, can you define a process that is more complex? For example for each review finding, do X. Or go from end-to-end test, back to build.
2) In your setup, how does a sub-agent prove undeniably, that it's work is complete? Does the "lead" agent just look at the output? If so, it would effectively make the lead an implicit reviewer for all agents, so I don't follow why you would need a review step.
3) Can you have steps in between these agentic processes that do not involve agents?
For 1), yes, there is an "observe" step in the process where - when the project is deployed - it observes and reconciles what happens vs what should happen based on specs.
I believe more variant are bound to emerge when harnesses become more prevalent. We only scratched the surface, so don't generalize over the process yet.
It took me awhile to figure out what this even does, and that was only because our team had just discussed agent coordination the day prior.
There are AI people who live and breathe AI every day, and people who don't know AI at all. In-between are lots of people in the middle of "leveling up" with AI ... but ATM the tool's docs are only for the very first group.
FWIW, I really think a paragraph or so of plain English describing the problem (agents can't communicate by default) and the solution (this tool lets them coordinate) would help everyone else (and would take like two seconds to add).
This looks really cool. Did you think about including automatic worktree creation + sandboxing?
I've built sth similar (more focus on the project setup and being able to work on multiple things at once with a single agent), that uses git worktrees to create a separate space (symlinks .env files) and bubblewrap to isolate the worktree for the agent.
We have been working on a communication layer that would be, I believe, complementing it by allowing the agents to actually talk to each other and to agents in other teams: https://github.com/awebai/aweb
How are you finding the communication between your agents? Ours is really interesting - they are developing personalities. Sometimes being quite passive aggressive with each other
Hmm... they remain similar-ish. But I think that's also because I have a lot in my claude.md file that they have to follow empiricism as a philosophy. Things need to be verified, researched, backed by data, etc.
That's also how I live my life, so when it starts to develop a personality, usually I'd ask what the benefit of that is. If it doesn't have a good answer, then I have yet another rule in my CLAUDE.md about what it should or should not do.
The communication, more or less, goes as I want it to go, aka quite smooth.
That's really interesting - so they develop personalities if we allow them to?! Is it part of social conditioning? Would humans have a personality if they lived in isolation?
When an agent is told to do something by the scheduler, the next step in the process only believes it’s done if the agent’s status is marked as ‘posted’. Statuses like ‘ready_to_post’ or ‘draft_verified_awaiting_review’… these are actually errors that the system needs to fix on the following attempt.
The trickiest part was dealing with being stopped, but not having something break. You have to have ways to say “this happened, and it isn't what we wanted”, for example, ‘blocked_quota’, ‘blocked_no_credentials’, or ‘skipped_anti_bunching’. If you don't have those, the main program will endlessly retry and spend all your money.
the typed handoff in ahk is the right primitive imo. discipline on top: agents never write half-states. every run terminates in a documented terminal status, success or otherwise.
Two questions on the design: "lead-agent reads sub-agent output" makes the lead an LLM judge. Are there post-conditions checked, or is LLM-judge approval the final gate? Related: does the system reason over typed state or over raw output?
Interesting. Sadly my super biased personal issue with any tool that needs nodeJS and its infrastructure is an immediate downer. Last tools like openspec and oh-my-codex etc I tried via a nix shell. Will take a look.
1) Can you define a process other than build -> review -> .. etc. And more importantly, can you define a process that is more complex? For example for each review finding, do X. Or go from end-to-end test, back to build.
2) In your setup, how does a sub-agent prove undeniably, that it's work is complete? Does the "lead" agent just look at the output? If so, it would effectively make the lead an implicit reviewer for all agents, so I don't follow why you would need a review step.
3) Can you have steps in between these agentic processes that do not involve agents?
For 1), yes, there is an "observe" step in the process where - when the project is deployed - it observes and reconciles what happens vs what should happen based on specs.
I believe more variant are bound to emerge when harnesses become more prevalent. We only scratched the surface, so don't generalize over the process yet.
There are AI people who live and breathe AI every day, and people who don't know AI at all. In-between are lots of people in the middle of "leveling up" with AI ... but ATM the tool's docs are only for the very first group.
FWIW, I really think a paragraph or so of plain English describing the problem (agents can't communicate by default) and the solution (this tool lets them coordinate) would help everyone else (and would take like two seconds to add).
I've built sth similar (more focus on the project setup and being able to work on multiple things at once with a single agent), that uses git worktrees to create a separate space (symlinks .env files) and bubblewrap to isolate the worktree for the agent.
We have been working on a communication layer that would be, I believe, complementing it by allowing the agents to actually talk to each other and to agents in other teams: https://github.com/awebai/aweb
I have a lot more roles though and it's more flexible, but also a bit slower as it isn't in full yolo mode.
That's also how I live my life, so when it starts to develop a personality, usually I'd ask what the benefit of that is. If it doesn't have a good answer, then I have yet another rule in my CLAUDE.md about what it should or should not do.
The communication, more or less, goes as I want it to go, aka quite smooth.
When an agent is told to do something by the scheduler, the next step in the process only believes it’s done if the agent’s status is marked as ‘posted’. Statuses like ‘ready_to_post’ or ‘draft_verified_awaiting_review’… these are actually errors that the system needs to fix on the following attempt.
The trickiest part was dealing with being stopped, but not having something break. You have to have ways to say “this happened, and it isn't what we wanted”, for example, ‘blocked_quota’, ‘blocked_no_credentials’, or ‘skipped_anti_bunching’. If you don't have those, the main program will endlessly retry and spend all your money.
the typed handoff in ahk is the right primitive imo. discipline on top: agents never write half-states. every run terminates in a documented terminal status, success or otherwise.
How does this differ from RooCode and similar agent orchestration tools?