Spatial AI agents: the infinite canvas for orchestration

Spatial AI agents are autonomous agents that operate on an infinite visual canvas instead of a linear chat box, giving human operators real-time spatial observability into what every agent is doing. By placing agents on a two-dimensional plane, teams can watch a multi-agent swarm coordinate, intervene mid-task, and govern autonomous work that logs alone keep hidden in a black box.

Spatial AI agents represent a fundamental shift in how autonomous systems interact with human collaborators and organizational data. The first wave of enterprise AI was dominated by the chat interface - essentially a narrow, linear pipe of text - but a new architectural paradigm is emerging that uses the infinite canvas as a primary control surface. This evolution addresses one of the most significant hurdles in agentic workflows: the lack of visual observability. When agents operate in a vacuum, hidden behind logs or API calls, trust and governance become impossible to maintain at scale - the same missing observability layer that has stalled so many production deployments.

Our research into spatial interaction models, particularly through the lens of open-source SDKs like tldraw, suggests that the canvas is not just a whiteboarding tool. It is becoming the primary operating surface for multi-agent swarms. By moving agents onto a two-dimensional plane, we solve the spatial reasoning gap that has historically limited LLMs, while giving human operators a real-time, high-level overview of complex autonomous work - a visual approach to managing AI agents through orchestration rather than dashboards.

The spatial reasoning gap in LLM architecture

To understand why spatial AI agents are necessary, we must first acknowledge a core limitation of current large language models (LLMs). These models are natively trained on text - a one-dimensional sequence of tokens. While they can generate remarkably coherent prose or functional code, they are historically poor at understanding 2D spatial relationships. If you ask a standard LLM to align five UI elements or describe the visual layout of a complex diagram, it will frequently hallucinate coordinates or fail to grasp the physical proximity of objects.

Closing this gap requires more than just a better prompt; it requires a spatial harness. This harness combines visual screenshots with JSON-based data structures that represent the canvas state. In early research projects like the "teach" initiative, researchers found that by providing an LLM with both a rasterized view of the canvas and a structured data layer, the model could begin to "see" the environment.

For example, in a test scenario where a user asked an agent to make a cartoon mouse blow out a candle, the agent had to interpret the relative positions of the mouse and the candle, generate new shapes for wind and smoke, and place them with pixel-perfect accuracy. This wasn't a pre-programmed animation - it was an agentic reaction to a spatial environment. This capability is the foundational layer for any agent that needs to manipulate design files, manage logistics maps, or navigate a desktop environment.

Why spatial AI agents need visual observability

For CTOs and internal AI champions, the biggest challenge in deploying agents is knowing what they are doing while they are doing it. In a traditional text-based or background-process model, the only way to monitor an agent is to tail a log file or wait for a final output. This creates a "black box" problem that prevents intervention and erodes user trust. Spatial AI agents solve this through visual observability: when agents are represented as physical entities on a canvas - often referred to as "fairies" or avatars in research environments - their intent and status become immediately apparent.

Consider the following operational benefits of a spatial interface:

• Real-time status tracking: Instead of reading a status string like executing_task_3, a manager can see the agent's avatar positioned next to a specific node in a dependency graph.
• Differentiated identity: By using visual traits (such as different colors, hats, or even movement styles), teams can distinguish between an orchestrator agent and a worker agent at a glance.
• Spatial grouping: When agents are collaborating on a specific project, they naturally cluster in a specific area of the canvas. This lets a human observer understand the organizational structure of a swarm without opening a single management dashboard.

One of the most compelling findings from our research is that animations - often dismissed as aesthetic "fluff" - are actually critical for governance. A bored agent that is waiting for a process might idle or wander, while an active agent might pulse or move between data nodes. This ambient information allows a human lead to scan a canvas with fifty agents and immediately identify bottlenecks or stalled processes.

The multi-agent coordination layer

As organizations move from single-purpose bots to complex, multi-agent systems, the problem of coordination becomes paramount. We have observed that the canvas serves as a superior "group chat" for agents. In a spatial environment, an orchestrator agent can assign tasks by physically moving data objects to different worker agents.

In one research demo involving board meeting preparation, an orchestrator agent was given a high-level goal: "Prepare a financial memo for fiscal year 2025." The orchestrator didn't just dump a wall of text. It created a visual plan on the canvas, assigned sub-tasks to other agents (one for data retrieval, one for chart generation, one for drafting), and monitored their progress spatially.

This creates a collaborative loop where humans can intervene. If a human sees an agent drafting a memo with the wrong fiscal assumptions, they can literally "grab" the agent or the data object and move it to a different section of the canvas, or leave a spatial comment that the agent can read and react to in its next turn. This level of granular, mid-process control is nearly impossible in a traditional autonomous loop.

Beyond the canvas trap: connecting to the real world

The most significant critique of early canvas-based AI is the "canvas trap" - the idea that agents are only useful for drawing shapes. To be enterprise-ready, spatial AI agents must break out of the SDK and interact with the real world.

We are now seeing the emergence of "Tech Tree" applications. In this model, the canvas acts as a dependency graph for real-world engineering tasks. Each node on the canvas represents a GitHub pull request, a Notion document, or a local file operation. See how a governed code review agent turns those canvas nodes into real, audited engineering outcomes rather than throwaway diagrams.

Key capabilities of this real-world integration include:

• Desktop control: Agents using the canvas as a window manager, resizing and moving real application windows (like Slack or Chrome) based on spatial logic.
• Cross-platform retrieval: An agent can be prompted to find a specific Notion spec mentioned in an email and then implement that spec as code in a local repository, all while reporting its progress visually on the canvas.
• Ephemeral UIs: Agents can generate temporary buttons, sliders, or input fields on the canvas to ask for human clarification, then delete them once the task is complete.

This transition from "toy" to "tool" is what makes spatial AI a viable infrastructure choice. It allows the canvas to become the high-level management layer for a company's entire technical stack.

Infrastructure implications: why state and sovereignty matter

For the CTO making architectural decisions, spatial AI agents require a different kind of infrastructure. You cannot run these systems on ephemeral, stateless "wrappers." They require a persistent, governed environment where state is shared between multiple users and multiple agents.

This is where the concept of a sovereign managed instance becomes critical. Because these agents are interacting with sensitive internal data - your emails, your codebases, your financial projections - the "place" where they work must be secure. A sovereign instance provides the audit logs, role-based access control (RBAC), and tenant isolation these agents need, running on infrastructure you own or as a dedicated instance operated for you - never a shared multi-tenant silo. This is the model Trinity is built on: the runtime for production agents, with the data sovereignty procurement departments demand behind it.

Unlike SaaS-based agent platforms that lock your data into a proprietary silo, a sovereign spatial infrastructure lets you own the "memory" of your agents. The canvas doesn't just show you what is happening now; it becomes a historical record of every decision, every movement, and every collaboration that occurred during a project's lifecycle.

Conclusion: the canvas as company infrastructure

The move toward spatial AI agents represents a shift from "AI as a tool" to "AI as a coworker." By leveraging the infinite canvas, organizations can finally solve the twin problems of LLM spatial reasoning and agent observability. This isn't about making AI more "fun" - it's about making it more operable, manageable, and trustworthy for the teams that rely on it.

As we look toward 2026, the companies that lead the market will be those that move their agents out of the black box and onto the map. Whether you are managing a complex software launch or orchestrating a global supply chain, the ability to see your agents work - and to work alongside them in a shared space - is the key to true autonomous transformation. The infrastructure you choose to host these agents will determine whether they are "trapped" in a silo or empowered to drive real-world outcomes.