The Gateway — a LangGraph deep agent

At a glance

How a turn works

Each message from the IDE becomes one turn on the graph. The gateway drains the LangGraph event stream and forwards frames to the plugin as they happen, so you watch the agent think, call tools and write code live:

Human-in-the-loop, server side

HITL is implemented as a real LangGraph interrupt. When the agent is about to take an action that needs approval, the gateway pauses the graph, sends an interrupt frame (with a preview of what it wants to do) and holds the turn. The plugin shows the prompt; your decision returns as a resume frame carrying a Command(resume=…), which continues the same turn from exactly where it paused — no replay, no lost context.

Whether HITL is active is decided per thread by the Auto toggle the plugin sends: Auto off enables HITL (the gateway interrupts for approval), Auto on disables it (the graph runs without pausing). The gateway stores that (and the redaction preference) on the thread's record, applied on every frame.

One exception overrides the Auto setting: a fixed set of high-risk shell commands — git, sudo and docker — always triggers an approval interrupt, even when Auto is on. The gateway detects these commands before running them and pauses the turn regardless of the thread's HITL preference, so destructive or system-level actions can never run unattended.

Threads & identity

The gateway keeps an in-memory ThreadRegistry: one record per conversation holding its message history, HITL/redaction settings and the machine identity (machine_id, OS, arch, hostname) the plugin sent in its hello frame. Because state lives on the record and not in a request-scoped variable, threads survive across turns and can be rehydrated after a reconnect.

Routing is purely by thread_id against a single gateway hub, which is what makes multi-user and multi-machine setups work: two agents on different machines are just two threads on the same hub. Serving several projects in parallel from separate workstations is already tested; the first release rolls out to a limited number of users while our infrastructure scales to a full multi-user deployment.

Secret redaction

Multi-agent coordination (ACP)

Real work often spans several projects — a backend and the library it consumes, for instance. The Gateway includes an Agent Coordination Protocol (ACP) so one agent can ask the agent working on another project to make a change, then continue once it reports back.

• Turn-based, not blocking. acp_request delivers a masked request that starts a fresh, independent turn on the peer agent; the requesting agent's turn simply ends.
• Report on completion. The peer's acp_report is delivered as a new turn on the original agent, which then resumes and finishes the task.
• Cache-until-idle. If the target is busy or offline, the request/report waits and is redelivered when it next goes idle or reconnects — even across machines.

This design deliberately avoids graph-level suspension, so no node is ever replayed and each side runs one clean turn.

Bring your own provider

The gateway builds its LLM through a single multi-provider factory, so the very same agent can run on OpenAI, Anthropic, Google or a fully local Ollama server just by changing configuration. With Ollama the whole stack is offline.

Where your keys live — the OS keystore

Your provider API keys are never baked into the plugin jar or persisted on the gateway. The PyCharm plugin stores each key (Anthropic · OpenAI · Google) in the operating system's keystore via IntelliJ's PasswordSafe — that means the native credential store on your own machine (macOS Keychain, Windows Credential Manager, or the Linux Secret Service / KeePass fallback).

• Stored on your machine. Keys sit in the OS keystore under your user account — not in plaintext config, not in the plugin binary.
• Provided per turn. When you send a message, the plugin reads the key from the keystore and hands it to the gateway inside that request; the gateway uses it to build the model client for that turn and does not store it.
• Only the provider you chose ever sees it. With a local Ollama endpoint no key leaves your infrastructure at all.

Local models need a GPU too — VRAM & RAM matter

Running fully offline with Ollama means the model itself runs on your host, and that is GPU work. Like the RAG's embedding phase, Ollama is far more capable with a GPU behind it — NVIDIA hardware is the recommended and tested path.

• Bigger VRAM = bigger / better models. The amount of GPU VRAM on the Ollama host directly caps which models you can load and how fast they run — more VRAM lets you run larger, more capable local models fully offline. See server configuration for details.
• System RAM helps too. Ample host RAM lets larger models and longer contexts stay resident, especially when layers spill from VRAM.
• Sharing a host. If Ollama and the RAG run on the same machine, size the GPU/VRAM and RAM for both — the more headroom, the better the fully-local experience.

Where it fits

See the full architecture →