Great. For my game [1] I have long had the capacity to compile a separate commandline application using unity code [2]. This lets me do benchmarks and unit tests on the core chess AI code without the editor. However I've had to maintain a separate second unit test suite that I can only run in the editor manually if any unity specific code is involved. Naturally, it's run far less often.
This new unity cli doesn't replace that design, since it's important to me that my core chess variant AI code remains a project on its own, totally independent of unity. But it does mean lots of my utilities like my second unity test, which normally require clicking a button in the editor, can now be run without the editor. Excellent!
Making a 2d chess variant game means almost all unity editor features just get in my way.
What I really want is the ability to do more operations which are decoupled from the editor. Even just being able to reliably type check Unity C# code without a running editor would be great, or running unit tests. Would really improve things like CI as well as the ability to run parallel development sessions.
Headless execution exists and is one of the simplest ways to get at this experience.
The biggest issue with unity is that the editor (even when headless) is designed to take an exclusive lock over the physical project directory. This can be solved by doing a robocopy of the project to a temporary path, deleting any lock files, and then running headless execution there. You can keep the same temp path(s) over many iterations with incremental sync so it's not copying the entire universe each time.
I did that for my Unity games like a decade ago, you can too :) Requires a design that has one core and then using Unity basically as a renderer/viewer though, but is a relatively easy design actually. And yes, automation and DX gets so much easier. I think Captain of Industry is built like this too, remember reading about it here on HN from the developers in some comments.
Same here, in Godot! This is a common practice for games with performance/networking constraints too. For example building a deterministic sim in a regular engine tends to fail without custom primitives or a nice library to handle some of the intricacies. By moving non-display code out of the engines you can reliably build and control all of this.
Will be exploring this but so far using a Unity MCP has proven mostly enough to do pretty much anything from outside.
I guess the build pipeline part is nice as we can finally have a fully automated build process from build to distribution without finnicky build/postbuild scripts/profiles.
> The same surface is open to any agent, whether that's Unity's own tooling routing through its MCP server, a third-party assistant, or a script you write yourself. Unity’s AI is one consumer of it, not a replacement for it.
I hope this means games get more automated tests! The state of the industry is generally woeful in that regard.
seems like agent-native CLIs are going to become a thing, where machine-readable, token-efficient output and discoverability matter just as much as human ergonomics
> unity command eval runs C# code live inside a running Editor or Player and returns the result, with no project-level recompile or domain reload required.
I played around with automating Unity with LLMs for a while. The ability to do this would have been very compelling to me at the beginning.
At this point, I think the more productive way for frontier models to interact with a unity project is to directly manipulate the raw scene files. They are not binary. These files are effectively YAML documents that you can parse, grep and patch just like any other text/code document. The key is to build the tools around these in the same way you would build tools around an agent that can read and manipulate multi-megabyte json documents. It's the same pattern.
Working with a unity project from the filesystem is so much simpler than working with one through all the APIs/CLIs/etc. The files in the unity project path represent 100% of the required information, so the editor feature surface is not relevant unless you are explicitly trying to build editor tooling.
I spent a solid month fighting the domain reload dragon only to realize it didn't fucking matter. I can just wait for the domain reload. As long as the work iteration step size is meaningful, this is not a huge deal.
> I think the more productive way for frontier models to interact with a unity project is to directly manipulate the raw scene files.
I disagree. Even moreso than webdev, gamedev benefits from a fast and reliable loop, especially one that can capture realtime state and screenshots. It's the difference between having a new well tested mechanic in your game next morning and giving up after a week of debugging.
Agents will work around slow iteration loops perfectly fine given enough time, but when you're shipping 20 core things in your game each day, the agent's test harness becomes the bottleneck.
IMO it's worth it to spend a day or two to make sure your test loop is tight, it will pay dividends. Not saying Unity is great at playing ball here though. Hopefully their new effort helps.
Letting the agent take screenshots of the scene is not effective. A human needs to be in that loop.
You can get primitive layouts working if you feed it orthographic projections from the 6 faces, but it falls apart once you need to start looking at things in perspective. It's also absolutely ass at interpreting scene illumination.
We need models trained on 3d scene transforms (aka world models). Passing raster images into the model is a really inefficient method of sending this logical information.
This new unity cli doesn't replace that design, since it's important to me that my core chess variant AI code remains a project on its own, totally independent of unity. But it does mean lots of my utilities like my second unity test, which normally require clicking a button in the editor, can now be run without the editor. Excellent!
Making a 2d chess variant game means almost all unity editor features just get in my way.
[1] https://www.chesscraft.ca
[2] https://youtu.be/U4xBLTpZ9Xo?si=Gd2aAWKvLqBVBU66
The biggest issue with unity is that the editor (even when headless) is designed to take an exclusive lock over the physical project directory. This can be solved by doing a robocopy of the project to a temporary path, deleting any lock files, and then running headless execution there. You can keep the same temp path(s) over many iterations with incremental sync so it's not copying the entire universe each time.
This works for CI, LLM automation, etc.
I guess the build pipeline part is nice as we can finally have a fully automated build process from build to distribution without finnicky build/postbuild scripts/profiles.
> The same surface is open to any agent, whether that's Unity's own tooling routing through its MCP server, a third-party assistant, or a script you write yourself. Unity’s AI is one consumer of it, not a replacement for it.
I hope this means games get more automated tests! The state of the industry is generally woeful in that regard.
I played around with automating Unity with LLMs for a while. The ability to do this would have been very compelling to me at the beginning.
At this point, I think the more productive way for frontier models to interact with a unity project is to directly manipulate the raw scene files. They are not binary. These files are effectively YAML documents that you can parse, grep and patch just like any other text/code document. The key is to build the tools around these in the same way you would build tools around an agent that can read and manipulate multi-megabyte json documents. It's the same pattern.
Working with a unity project from the filesystem is so much simpler than working with one through all the APIs/CLIs/etc. The files in the unity project path represent 100% of the required information, so the editor feature surface is not relevant unless you are explicitly trying to build editor tooling.
I spent a solid month fighting the domain reload dragon only to realize it didn't fucking matter. I can just wait for the domain reload. As long as the work iteration step size is meaningful, this is not a huge deal.
I disagree. Even moreso than webdev, gamedev benefits from a fast and reliable loop, especially one that can capture realtime state and screenshots. It's the difference between having a new well tested mechanic in your game next morning and giving up after a week of debugging.
Agents will work around slow iteration loops perfectly fine given enough time, but when you're shipping 20 core things in your game each day, the agent's test harness becomes the bottleneck.
IMO it's worth it to spend a day or two to make sure your test loop is tight, it will pay dividends. Not saying Unity is great at playing ball here though. Hopefully their new effort helps.
You can get primitive layouts working if you feed it orthographic projections from the 6 faces, but it falls apart once you need to start looking at things in perspective. It's also absolutely ass at interpreting scene illumination.
We need models trained on 3d scene transforms (aka world models). Passing raster images into the model is a really inefficient method of sending this logical information.
Epic jumped on the AI bandwagon a while ago.
Unity has been the laggard in terms of AI adoption. If anything this is an "ending" to their old ways.
[1] https://www.tryaura.dev/