Hermes Agent Comes to the Desktop: Install, Optimization, and the Best Local Models

June 4, 2026 12 min read AI & Machine Learning

hermes-agentnous-researchagentslocal-llmollamaself-hostingagentic-workflowsdesktop-app

Contents

When Nous Research released Hermes Agent in February 2026, it was a terminal animal: a single curl command, a config file, and a chat loop in your shell. The pitch was unusual even by 2026 standards — an autonomous agent with a genuine learning loop that writes its own skills, refines them during use, and builds a deepening model of who you are across sessions. The catch was that you had to live in a TUI to use it. On June 3rd that changed: Nous shipped Hermes Desktop, a native cross-platform front end (currently build v0.15.2, in public preview) that was first demoed during Jensen Huang’s GTC keynote and brings the agent out of the terminal without changing what it is underneath.

This post is the practical companion to that release. It covers what the desktop app is and how it relates to the CLI, how to install both, how to wire Hermes to fully local models through Ollama, which local models are actually worth running for agentic work in 2026, and how to tune the agent so its memory and skills help rather than get in the way. If you want the broader framing — what Hermes is and how it stacks up against OpenClaw — the earlier Hermes Agent setup and comparison covers that ground; this one is about the desktop app and getting the most out of local inference.

What Hermes Desktop Actually Is

The most important fact about Hermes Desktop is what it is not: it is not a separate product or a lightweight clone of the agent. It is a graphical shell on the exact same agent core that powers the CLI, the TUI, and the gateway. Sessions, configuration, API keys, skills, and memory are fully interchangeable across all four surfaces. A skill the agent learned while you were driving it from the terminal is available when you open the desktop app, because both read the same HERMES_HOME directory (~/.hermes on Linux/macOS, %LOCALAPPDATA%\hermes on Windows). The packaged installer simply drops the Hermes runtime into that same layout, which is why the two are interchangeable rather than merely compatible.

What the GUI adds is everything the terminal made awkward:

Streaming responses with live tool activity. You watch the agent think, call tools, and stream output in real time, with structured summaries instead of a wall of log lines.
A right-hand preview rail. Web pages the agent fetches, files it touches, and tool outputs render in a side panel next to the chat rather than scrolling past in text.
A file browser for exploring the working directory without leaving the app.
Voice mode for audio input and output (macOS prompts for microphone permission on first use).
Drag-and-drop file attachment straight into the chat.
A settings UI for providers, models, tools, and credentials — the config you used to hand-edit.
Management panes for skills, cron jobs, profiles, and multi-agent orchestration, which is the part that turns it from a chat window into a control surface for a fleet of agents.

Here is how the pieces relate:

        ┌──────────────────────────────────────────────┐
        │            Hermes Agent Core                  │
        │  learning loop · skills · memory (FTS5) ·     │
        │  user model (Honcho) · sandbox · tool gateway │
        └───────────────┬──────────────────────────────┘
                        │  same HERMES_HOME, same config
   ┌──────────┬─────────┼─────────────┬──────────────┐
   ▼          ▼         ▼             ▼              ▼
 ┌─────┐  ┌───────┐  ┌──────────┐  ┌─────────┐  ┌──────────┐
 │ CLI │  │  TUI  │  │ Desktop  │  │ Gateway │  │  Web     │
 │     │  │       │  │  (GUI)   │  │ (--tui) │  │dashboard │
 └─────┘  └───────┘  └────┬─────┘  └────┬────┘  └──────────┘
                          │             │
                   local agent    remote backend over
                   on this box    /api/ws WebSocket

That last detail matters for homelabs: the desktop app can drive a Hermes instance running on another machine. Under Settings → Gateway → Remote gateway you point it at a remote URL and session token, and the backend has to be started with the --tui flag so it exposes the /api/ws WebSocket endpoints the desktop chat needs. So you can run the heavy agent (and the heavy local model) on a GPU box in the closet and drive it from a laptop.

Installing Hermes

There are two things to install — the agent core and, optionally, the desktop front end — but the install paths fold them together.

The agent (and optionally the desktop) in one shot. On Linux, macOS, or WSL2:

1

curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash

On Windows via PowerShell:

1

iex (irm https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.ps1)

To bundle the desktop app with that same install, add --include-desktop to the script invocation. The Android/Termux path uses the same Linux curl command with auto-detection. (As always with curl | bash, read the script first if you don’t trust the source — it is open on GitHub.)

The desktop app on its own. On macOS and Windows the recommended route is downloading the native installer from hermes-agent.nousresearch.com/desktop. If you already have a CLI install, you don’t need to download anything — just run:

1

hermes desktop

and it launches the GUI against your existing configuration, providers, and skills.

First-run setup. After install, the fastest way to a working agent is the portal flow:

1

hermes setup --portal

This runs an OAuth handshake that provisions a model plus four Tool Gateway tools — web search, image generation, text-to-speech, and a browser. That gets you a capable hosted agent in about a minute. But the reason most people reading this are here is to not depend on a hosted model, so let’s wire it to local inference.

Going Fully Local with Ollama

Hermes was designed to run against local models as a first-class case, not an afterthought, and the integration with Ollama is the smoothest path. There is a one-liner that does the entire dance — install Hermes, configure the Ollama provider, and set your downloaded model as primary:

1

ollama launch hermes

If you’d rather do it by hand (or you already have Hermes installed), run hermes setup and configure a custom provider:

Choose “More providers…” then “Custom endpoint (enter URL manually)”.
Enter Ollama’s OpenAI-compatible endpoint:
```
http://127.0.0.1:11434/v1
```
Leave the API key blank — local Ollama doesn’t need one.
Hermes auto-detects your downloaded models and asks which to use.
Leave context length blank for auto-detection.
Confirm, and the chat starts.

You can reconfigure at any time by re-running hermes setup. The same custom-endpoint approach works for any OpenAI-compatible local server — point it at a Jan server on :1337, an LM Studio endpoint, or a vLLM instance — but Ollama is the documented happy path with per-model tool-call parsers tuned for local quirks.

Choosing the Right Local Model

This is the decision that makes or breaks a local Hermes setup, and it is not about raw benchmark scores. An agent lives or dies on tool calling — its ability to emit a correct, parseable function call and then act on the result — and on context length, because the agent stuffs its memory, skills, and conversation history into every prompt.

Two hard constraints frame the choice:

Hermes requires at least a 64K context window. Most modern models clear this easily, but it rules out older 8K and 16K models entirely.
Tool-calling reliability is the dominant axis. A model that reasons beautifully but botches function-call syntax is useless as an agent. This is why small models disappoint here even when their chat quality seems fine.

Here is the practical 2026 shortlist:

Model	VRAM (Q4_K_M)	Why pick it
Llama 4 Maverick	Multi-GPU / large	Best open-source tool calling as of spring 2026; ~1M context; approaches cloud quality for structured calls
Qwen3.5 27B	~16 GB	The value sweet spot: reliable tool calls, strong reasoning, fits a single 24 GB card
Mistral Small	~12-16 GB	Second-best function calling at a smaller size; good when 27B won’t fit
Hermes 4 (70B)	40+ GB	Nous’s own model, co-designed with the agent; best alignment between agent loop and model behavior
Gemma 4	~16 GB	Strong general reasoning and code; solid function-calling variant

The selection logic is simple. If you have a 24 GB consumer card (RTX 3090/4090/5090), Qwen3.5 27B at Q4_K_M is the default recommendation — it fits, it calls tools reliably, and it reasons well enough for real workflows. If you have more headroom, Hermes 4 70B gives you the tightest integration because Nous tuned the agent and the model family together. If you’re tight on VRAM, Mistral Small is the graceful fallback. Reach for Llama 4 Maverick only if you have the multi-GPU hardware to host it; below that tier its 1M context is theoretical. For the quantization tradeoffs behind these Q4_K_M choices, see the quantization deep dive, and for getting the models themselves running well, self-hosting Llama 4 and Gemma 4.

A blunt warning: anything under roughly 14B will struggle to run Hermes as a real agent. It may chat fine, but it will drop tool calls, mangle JSON arguments, and lose the thread inside a 64K context full of memory and skills. Local agentic work has a hardware floor, and it is higher than local chat.

Optimization and Tuning

Once a capable model is connected, the gains come from three places: VRAM utilization, the memory/skill system, and profiles.

Squeeze the context into VRAM. If you serve the model through a backend that exposes it (vLLM, or Ollama’s newer flags), raising GPU memory utilization buys you context headroom. The agent is context-hungry, so the difference between a 32K and a 96K effective window is the difference between an agent that remembers the task and one that forgets it mid-run:

--gpu-memory-utilization 0.95    # default is 0.9; reclaims VRAM for KV cache

Manage the learning loop, don’t just let it run. Hermes’s differentiator is that it autonomously creates skills from experience, improves them during use, nudges itself to persist knowledge, and recalls past sessions via FTS5 full-text search with LLM summarization. This is powerful and it is also a source of drift: left unattended, an agent accumulates half-baked skills and stale memories. Treat the skills pane in the desktop app as something you curate — prune skills that don’t earn their keep, and review what the agent has learned about you (the user model is built via Honcho’s dialectic approach) the way you’d review a config file. Skills are portable and shareable through agentskills.io, so good ones can be exported rather than relearned.

Use profiles as trust boundaries. This is the single most important operational habit. A profile is a clean boundary for credentials, tools, skills, and memory. If a workflow has different trust requirements — say, one agent that can touch your email and another that runs untrusted code — give each its own profile instead of mixing everything into the default environment. The desktop app’s profiles pane makes this a click rather than a config edit.

Match the sandbox to the risk. Hermes can execute its work in several backends: local, Docker, SSH, Daytona, Singularity, and Modal. The local sandbox is the weakest isolation — the agent runs commands directly on your machine — which is fine for trusted, read-mostly tasks and a bad idea for anything that executes untrusted code. Use Docker for a real boundary on your own hardware, or the serverless Daytona/Modal backends, which hibernate when idle so they cost almost nothing between runs. Running an autonomous agent with shell access is inherently risky; the sandbox choice is your main lever for containing it. For the broader pattern of building agents that don’t go off the rails, building AI agents that work and homelab AI agents are worth reading alongside this.

Hardware and Honest Trade-offs

A rough sizing guide for self-hosting the model behind Hermes:

Tier	RAM/VRAM	What you can run
Entry	8 GB	7-8B models — chat only, unreliable as an agent
Workstation	16-24 GB	27-32B at Q4 — the real local-agent sweet spot
Serious	40-80 GB	70B — best local agent quality
Multi-GPU	80 GB+	Llama 4 Maverick-class with very long context

And the downsides you should walk in knowing:

The desktop app is public preview. v0.15.2 is genuinely usable but it is early; expect rough edges, and don’t bet a production workflow on it without a fallback to the CLI.
The self-improvement loop adds overhead. Skill creation, memory summarization, and the periodic nudges consume tokens and add latency on every interaction. On a local model that overhead is felt more sharply than against a fast cloud endpoint.
Local quality trails frontier cloud models. Even Qwen3.5 27B and Hermes 4 70B are a step behind a hosted frontier model on the hardest tool-use and long-horizon reasoning. Local buys you privacy and zero marginal cost, not parity.
Remote gateway is a security surface. The --tui WebSocket endpoint that lets the desktop drive a remote backend is something to firewall and authenticate carefully — it is, by design, a remote-control channel into an agent with shell access.
Memory and user-modeling are moving parts. The Honcho-based user model and the FTS5 recall are differentiators, but they are also extra subsystems that can misbehave; budget time to understand them.

The Verdict

Hermes Desktop is the release that makes the agent approachable. The terminal version was always powerful, but watching tool calls stream in a real UI, browsing the files the agent touched, and curating its skills from a pane instead of a config file lowers the barrier from “infrastructure project” to “app you open.” Crucially, none of that came at the cost of the thing that makes Hermes interesting — it is the same self-improving core, just with a face.

For local-first operators the recommendation is concrete: install with --include-desktop, wire it to Ollama with ollama launch hermes, and run Qwen3.5 27B at Q4_K_M if you have a 24 GB card or Hermes 4 70B if you have the VRAM for it. Then spend your tuning effort where it pays off — profiles as trust boundaries, a real sandbox for anything that runs code, and active curation of the skills and memory the agent generates. Do that, and you have an autonomous agent that runs entirely on your own hardware, learns from your work, and now finally has a window you can actually watch it through.