How the SDK integrates with Hermes Agent
The SDK treats Hermes Agent as an external dependency and needs no changes to Hermes. Everything goes through surfaces Hermes already publishes for third-party platforms. This page lists those surfaces, explains how each one is used, and states plainly where the SDK relies on behavior that is not a formal API. It ends with the few small upstream changes that would remove those workarounds.
The integration point: a platform plugin
Hermes's gateway runs "platform adapters": Telegram, Slack and about twenty others. A directory plugin with kind: platform can register a new adapter through ctx.register_platform(...) without touching core code; see gateway/platforms/ADDING_A_PLATFORM.md in Hermes. The SDK's plugin/ directory is such a plugin:
| Plugin file | Role |
|---|---|
plugin.yaml |
Manifest: kind: platform, optional env vars surfaced in hermes config |
__init__.py |
register(ctx): platform entry, agent tools, hermes gadget CLI |
adapter.py |
GadgetAdapter(BasePlatformAdapter): the Hermes-facing side |
hub.py |
WebSocket server, auth, audio, images, actions (no Hermes imports) |
tools.py |
gadget_devices, gadget_display, gadget_ask, gadget_action, gadget_speak, gadget_play, gadget_widget, gadget_interpret |
playback.py |
What gadget_play may open, decoding a chunk at a time, and the playback task (see Playing audio) |
interpret.py |
Interpreter mode: its state, the translation request and the device message (no Hermes imports) |
store.py, status.py, protocol.py, audio.py, textfmt.py, imaging.py |
Support modules (no Hermes imports) |
dashboard/ |
The Gadgets tab of the Hermes web dashboard: manifest.json, the backend routes in plugin_api.py, and the tab's bundle in dist/, built from src/ (see The Gadgets tab) |
desktop/plugin.js, desk.py |
The Gadgets page of the Hermes desktop app, built from desktop-src/, and the queue that carries its cards and actions to the gateway (see The Gadgets page in Hermes Desktop) |
Each device is a direct-message chat whose chat_id and user_id are the device id. Everything Hermes does for a DM on any platform then applies unchanged: sessions and memory, /new, /stop, /voice, model routing, approvals and delivery.
Configuration
# ~/.hermes/config.yaml
plugins:
enabled: [gadget] # or: hermes plugins enable gadget
platforms:
gadget:
enabled: true
extra:
host: 0.0.0.0 # interface devices connect to
port: 8765
path: /gadget
speak_replies: true # spoken replies for devices with a speaker
# speech_queue_wait_s: 600 # how long a message waits for a busy device to be spoken
# speech_overflow_target: telegram # where a message that couldn't be spoken goes (a send_message target)
auto_home: true # first approved device becomes the gadget home channel
unauthorized_dm_behavior: pair
# heartbeat_s: 20
# max_utterance_s: 60
# opus_bitrate: 24000 # Opus speaker audio for devices that decode it; 0 = PCM16 only
# play_dirs: [~/Music] # folders gadget_play may read sound files from (see Playing audio)
# play_max_minutes: 60 # the longest one gadget_play playback runs
# dashboard: true # Hermes's sessions and status tiles on devices that draw them
# profiles: [work, home] # profiles gadgets may switch to, or "all"; off without it (see below)
# profile_faces: true # each profile's face or avatar on gadgets (docs/faces.md)
# battery_log: true # keep each device's battery readings in a CSV (see below)
# language: es # the language devices on `auto` show; default: Hermes's own
# stt_language: pt # the language gadget speech is transcribed in; default: auto (see below)
# tls_cert: /path/cert.pem # serve wss://
# tls_key: /path/key.pem
battery_log: true writes each paired device's battery readings to <hermes home>/plugin-data/gadget/battery/<device id>.csv: a row every 30 seconds, and at once when the power source changes, with the time, battery_mv, battery_percent, charging, external_power and battery_present. Devices that report no battery voltage or percentage add nothing. A file that reaches 8 MB moves to <device id>.1.csv, replacing the previous one. Use a log of a full discharge, from a full charge to shutdown with USB unplugged, to check or fit a board's voltage-to-percent estimate. It is off by default and the gateway needs a restart after changing it.
opus_bitrate (24000 by default) sends spoken replies as Opus to devices whose firmware decodes it (caps.speaker.formats). It needs libopus on the Hermes host (libopus0 on Debian and Ubuntu, opus on Arch and Homebrew), loaded through ctypes with no Python package; without it, or with opus_bitrate: 0, replies go out as PCM16 as before. At 24 kbps a reply takes about a tenth of PCM16's 256 kbps, which keeps playback smooth on a phone hotspot or a slow Wi-Fi link.
Secrets go in ~/.hermes/.env, following Hermes's rule that .env is only for secrets:
GADGET_ACCESS_TOKEN: optional shared token every device must present.GADGET_ALLOWED_USERS: comma-separated device ids admitted without pairing.GADGET_ALLOW_ALL_USERS=true: admit any device (development only).
If you set
GADGET_ALLOWED_USERS(orGATEWAY_ALLOWED_USERS), Hermes switches unknown senders from "pair" to "ignore". Keepunauthorized_dm_behavior: pairin the platform'sextraso new devices still receive pairing codes.
Fix the language of gadget speech
Hermes transcribes a gadget's voice with the stt settings of the profile that answers it. A profile without a language for its STT provider, such as stt.local.language, leaves the language to the provider's auto-detection, and a short clip in another language can come back transcribed as English. stt_language sets the language for gadget recordings only, whichever profile answers:
hermes config set platforms.gadget.extra.stt_language pt
Use the language code your STT provider takes, such as pt, es or de. A value that isn't a language code (for example Portuguese) is ignored with a warning in the gateway log, and each profile's own STT settings apply. The default, auto, leaves every profile's STT settings as they are. Voice notes from other platforms keep their profile's settings either way. The plugin sets the language through Hermes's pre_transcription plugin hook, which wins over stt.<provider>.language. Restart the gateway after changing it.
On a multiplexed gateway, set it in the profile that runs the gadget platform, usually default. Hermes fires the hooks of the profile that answers, so the language applies to a profile only when that profile has the gadget plugin enabled and its copy of the plugin is this version or newer (cloned profiles keep their own copy). When the configured provider fails and Hermes falls back to local whisper on its own, that fallback doesn't fire the hook, so it uses the profile's own stt.local settings.
Change the gadget port
The adapter reads platforms.gadget.extra when the gateway creates it. Editing the configuration or toggling the plugin does not rebind a running gadget listener. Restart the gateway to apply changes to the port, host, path, or TLS settings.
If another service uses the default port, 8765, choose a free port on the Hermes computer. For example:
hermes config set platforms.gadget.extra.port 9100
Restart the gateway service with hermes gateway restart. If you run hermes gateway run in a terminal, stop that process and start it again. Allow the new port through the host firewall when devices connect from other computers.
After the restart, run hermes gadget info and use the new URL to update each device's server address. For a local simulator, this example becomes hermes-gadget sim --url ws://127.0.0.1:9100/gadget. Devices configured with the old port cannot reconnect until their server address changes.
hermes gadget info prints the configured URL. It does not confirm that the running gateway has rebound to that address. Check the gateway's startup log for the listening port and confirm that a device reconnects.
The Gadgets tab in the Hermes dashboard
The plugin adds a Gadgets tab to the Hermes web dashboard (hermes dashboard). It lists every device this Hermes knows, with its board, firmware, when it was last seen, whether it is waiting to be paired, and, while the gateway runs, whether it is online and its battery. The tab changes nothing on the devices: pair, rename, forget and update them with hermes gadget as before, or pair from the Desktop page. Each connected device has a Save diagnostics report button, which saves the same file as hermes gadget diag (see Save a board report).
It needs nothing beyond the plugin: the tab appears once the plugin is enabled (plugins.enabled: [gadget]) and the dashboard has been restarted, since the dashboard mounts plugin routes when it starts. Without the dashboard/ folder the plugin works exactly as before.
Which process does what. The tab's backend, dashboard/plugin_api.py, runs in the dashboard's process, behind the dashboard's own login: Hermes refuses an unauthenticated request to /api/plugins/gadget/ before the plugin's code runs. The devices' connections are in the gateway's process. The two never call each other; they read the same files in <hermes home>/plugin-data/gadget/, like hermes gadget does:
| File | Written by | The tab reads |
|---|---|---|
devices.json |
the gateway, hermes gadget pair and forget |
name, board, firmware, last seen, pending pairing. Device keys are never returned |
status.json |
the running gateway | which devices are connected now, since when, and their battery readings (battery_percent, battery_mv, charging, external_power, battery_present); for the Desktop page, whether each is paired, its action names, and what its screen shows |
desk/inbox/, desk/done/ |
the Desktop page's routes, the tab's diagnostics route and hermes gadget diag, then the running gateway |
the outcome of a card or action the page sent (see The Gadgets page in Hermes Desktop), or a diagnostics report |
The gateway replaces status.json in one step (a temporary file, then a rename), so a reader never sees half of it. It rewrites the file within about half a second of a device connecting or leaving, of a change of pairing or profile, of a new battery percentage or power source, and of a change to what a screen shows (listening, thinking or speaking, the words heard, the reply), and at least every 30 seconds otherwise. A voltage change alone waits for the next write. A gateway that stops writes the file with no devices online; one that dies without doing so leaves a file older than 90 seconds, which the tab treats the same way. Either way the tab says the gateway is not running and shows no online state or battery.
| Route | Returns |
|---|---|
GET /api/plugins/gadget/devices |
gateway: running and updated_at of the status file; devices: id, name, board, firmware, last_seen, enrolled_at, pending, online, connected_at and battery (null when offline or not reported), and for the Desktop page paired, actions and activity (state, heard, reply, status) |
GET /api/plugins/gadget/pairing |
Devices showing a pairing code Hermes hasn't approved: id, name, board, code |
POST /api/plugins/gadget/pairing/{id}/approve |
Approves the code that device is showing, through Hermes's PairingStore, as hermes gadget pair does |
POST /api/plugins/gadget/devices/{id}/card?title=&text=&seconds= |
Shows a card on a connected, paired device's screen; waits up to 8 seconds for the gateway's answer |
POST /api/plugins/gadget/devices/{id}/action?name=&args= |
Runs one of the actions that device advertises, with args as a JSON object; waits up to 18 seconds |
POST /api/plugins/gadget/devices/{id}/diag |
A connected device's diagnostics report with the gateway's recent plugin log: file (a name to save it under) and text. The device needn't be paired; waits up to 15 seconds |
GET /api/plugins/gadget/info |
Whether the platform is enabled and the gateway running, and the local URL a gadget on this computer connects to |
The web tab uses only /devices. The other routes are for the Desktop page, and like every plugin route they sit behind the dashboard's login.
The bundle. dashboard/dist/index.js is built output. Its source is dashboard/src/index.jsx; npm run build:dashboard-tab in site/ rebuilds it with esbuild (pinned in site/package.json) as one IIFE that takes React and the UI components from the dashboard's plugin SDK (window.__HERMES_PLUGIN_SDK__) and bundles no library. npm test in site/ fails when the committed bundle differs from a fresh build of the source.
The Gadgets page in Hermes Desktop
The Hermes desktop app gets a Gadgets page: a sidebar row, a route at /gadgets, and an Open Gadgets command in the palette. Turn the plugin on in Capabilities › Plugins if the page doesn't appear. It has three parts.
- A gadget in the window. The page runs the device protocol itself, as a gadget on this computer with a round screen, a microphone and a speaker. It connects to the gateway on this computer, at its configured
host(or127.0.0.1when the gateway listens on all interfaces), enrolls with a key it keeps in the page's storage, and pairs like any device: its code shows on its screen, and Approve next to it approves it. Hold the button or Space to talk, or type. Hermes answers on its screen and, when text-to-speech is set up, out loud. It advertises two actions the agent can call:light.set, a coloured glow around the screen, andtimer.start, a countdown that ends with a chime. It needs the Gadget platform on and the gateway running. When the gateway requires a token (GADGET_ACCESS_TOKENorextra.access_token), the page asks for it and keeps it in the page's storage. A card sent to stay until dismissed goes when you tap the screen. - Your gadgets, live. Each known device with a small copy of its screen: listening, thinking or speaking, what Hermes heard and the reply so far. The running gateway writes this to
status.json, and the page reads it about once a second. - Controls. Send a card to a connected device's screen, or run one of the actions it advertises, without asking the agent.
How a card reaches a device. The page's routes run in the dashboard's process and the device connections in the gateway's. The route writes a request to desk/inbox/ in the plugin's data folder and waits. The gateway checks that folder every 0.4 seconds, sends the card or action to the device, and writes the outcome to desk/done/. A request no gateway picks up within 30 seconds is dropped, and a route that gets no answer withdraws its request and says the gateway isn't running. The gateway skips a withdrawn request, so a card the page gave up on never shows late. At most 32 requests wait at once. A card's title is cut to 64 characters and its text to 600. Action arguments must be a JSON object of at most 2,000 characters. The gateway refuses a device that isn't connected, isn't paired, or doesn't advertise the action.
The bundle. desktop/plugin.js is built output. Its source is desktop-src/: npm ci && npm run build there rebuilds it with esbuild, and npm test runs the protocol client's tests (device id, authentication MAC and binary frames). Hermes Desktop publishes the file from the installed plugin. To try a change without reinstalling, copy it to ~/.hermes/desktop-plugins/gadget/plugin.js and reload the page.
Hermes surfaces the SDK uses
| Need | Hermes surface | Notes |
|---|---|---|
| Register a transport | ctx.register_platform(...) → PlatformEntry |
allowed_users_env, allow_all_env, platform_hint, max_message_length, parse_target_ref_fn |
| Setup wizard | PlatformEntry.setup_fn |
hermes gateway setup lists Hermes Gadget and runs its step: enable the platform, pick the port (written with hermes config set's writer), print the device URL and the installer link. The wizard then offers the gateway restart |
| Inbound messages | BasePlatformAdapter.handle_message(MessageEvent) |
text → MessageType.TEXT; voice → MessageType.VOICE with a WAV in media_urls |
| Store voice audio | cache_audio_from_bytes_async |
The gateway's own audio cache |
| Speech-to-text | Gateway STT for VOICE messages (stt.* config) |
Hermes transcribes; the device never runs ASR |
| Transcript echo | Gateway STT echo (stt_echo_transcripts) |
Forwarded to the device as transcript |
| Per-device context | MessageEvent.channel_prompt |
Stable text: screen size, speaker, action names. The ephemeral prompt is pinned per session, so prompt caching holds |
| Turn boundaries | on_processing_start / on_processing_complete |
Become turn.start / turn.end |
| Live status | supports_status_text = True + set_status_text() |
Becomes status frames |
| Streaming text | supports_draft_streaming() + send_draft(); edit_message() |
Used when streaming is enabled; becomes reply.delta |
| Final replies | send() |
Markdown stripped and folded to the device's charset (ascii, latin1 or latin1-cyrillic) for small screens |
| Whole-file TTS | play_tts() / send_voice() |
WAV decoded in Python; other formats through ffmpeg; resampled to the device rate |
| Streaming TTS | supports_streaming_tts / begin_ / write_ / finish_ / abort_streaming_tts |
PCM chunks forwarded as they are generated, for TTS providers Hermes can stream |
| Images from the agent | send_image_file() / send_image() |
Converted to RGB565 to fit the screen (needs Pillow) |
Confirmations (/new, /undo, model switches) |
send_slash_confirm() + tools.slash_confirm.resolve() |
Shown as a prompt; TALK resolves once, CANCEL cancel. The handler's reply goes back with send() |
| Dangerous-command approvals | _send_exec_approval_prompt(ExecApprovalPrompt) + tools.approval.resolve_gateway_approval_card() |
Shown as a prompt; TALK resolves once, CANCEL deny, for the request the prompt was shown for and no other (compat.resolve_approval(); a Hermes without the card path falls back to resolve_gateway_approval(), which answers the session's oldest). A command the screen can't show whole stays Deny-only in Hermes too (compat.approval_deny_only()). The prompt's ttl_s is Hermes's approval timeout (approvals.timeout, read by compat.approval_timeout_s()), so the device and the adapter both drop it when that time passes. On timeout Hermes calls edit_message() on the prompt, which withdraws it |
Agent questions (the clarify tool) |
send_clarify() |
Hermes's text fallback (the question with numbered choices; the next message, spoken or typed, is the answer), also spoken with Hermes's text-to-speech when the device speaks replies. clarify does not use the device's choice screen; the agent reaches that with the gadget_ask tool |
| Pairing | Hermes DM pairing (gateway/pairing.py, hermes pairing approve; hermes gadget pair approves through the same PairingStore) |
See below |
| Authorization state | BasePlatformAdapter._is_sender_authorized() |
The runner-installed check; used to mirror approval and revocation to the device |
| Agent tools | ctx.register_tool(toolset="gadget") |
Part of the implicit hermes-gadget toolset; also usable from other chats |
| Tool session context | gateway.session_context.get_session_env |
HERMES_SESSION_PLATFORM / HERMES_SESSION_CHAT_ID pick the default device |
| Durable state | plugins.plugin_storage.plugin_data_dir("gadget") |
Enrolled device keys, pending pairing codes, firmware staged for updates (updates/), and the running gateway's live device status (status.json) |
| Web dashboard tab | dashboard/manifest.json, dashboard/plugin_api.py (a FastAPI router under /api/plugins/gadget/), window.__HERMES_PLUGIN_SDK__ |
The Gadgets tab. Reads the plugin's data files; never calls the gateway. See The Gadgets tab |
| Desktop page | desktop/plugin.js with @hermes/plugin-sdk (ROUTES_AREA, SIDEBAR_NAV_AREA, PALETTE_AREA, ctx.rest, ctx.storage) |
The Gadgets page, on the same routes. See The Gadgets page in Hermes Desktop |
| Host CLI | ctx.register_cli_command("gadget", ...) |
hermes gadget devices / forget / info / pair / update. pair waits for a device to show a code and approves it after asking. update stages an image in the plugin's data directory (a file, or with --latest the newest release's image for the device's board, checked against the release manifest); the gateway's adapter installs it once the device is online and writes progress back for the command |
| Send and cron targets | parse_target_ref_fn, cron_deliver_env_var |
gadget:hg-0123456789abcdef works with send_message and cron delivery; GADGET_HOME_CHANNEL supplies a default cron device. On a device that speaks replies, such a message is also spoken (see Scheduled delivery) |
Scheduled delivery
Set a cron job's deliver to gadget:hg-0123456789abcdef, replacing the example ID with one from hermes gadget devices. The gadget must be paired and online, with the gateway running, to receive the scheduled message.
On a device that speaks replies, a message sent outside a turn, by cron or by send_message, is also read aloud with Hermes's text-to-speech: Hermes itself speaks only replies to voice turns, and a speaker with no screen would otherwise stay silent. Markdown is removed, and for cron output so are Hermes's header, footer and truncation note. Messages play one at a time per device, each after any audio already playing. A message that arrives during a turn, other than that turn's own reply (cron output, or send_message from another chat), is spoken after the turn. Nothing is dropped silently: a message not spoken within speech_queue_wait_s of arriving (600 s by default), or whose text-to-speech or playback fails, is sent with send_message to speech_overflow_target, such as telegram or discord:1234, prefixed with the device name and the reason. Without a target it is logged as a warning; the device has shown its text either way. The queue lives in memory, so a gateway restart loses messages still waiting. The caption of a voice file and the text of a clarify question are not spoken again. /voice off in the device's chat, or speak_replies: false, keeps these messages silent too.
To use deliver="gadget", configure the gadget home channel. By default, auto_home: true makes the first approved device the home channel. An explicit GADGET_HOME_CHANNEL device ID in the profile's .env overrides that default. A target with an explicit device ID does not require this environment variable.
Voice round trip
- Device: push-to-talk streams PCM16 to the hub at the microphone rate the device declares in
hello(24 kHz on ESP32 boards with codec chips, 16 kHz by default otherwise), and the hub assembles a WAV. - Adapter: caches the WAV with
cache_audio_from_bytes_async, then callshandle_message(MessageEvent(VOICE, media_urls=[wav])). - Gateway:
_enrich_message_with_transcriptionruns the configured STT provider. The transcript is echoed (device shows "> what you said"), then the agent turn runs. - Reply audio:
- When the TTS provider streams,
StreamingTTSConsumer→write_streaming_tts→ PCM to the device while text is still being generated. - Otherwise whole-file auto-TTS →
play_tts(file)→ decode → resample → paced PCM.
- When the TTS provider streams,
- Reply text:
send(text)→replyframe. Thenturn.end.
Because a gadget is voice-first, GadgetAdapter answers True from _should_auto_tts_for_chat for devices that declare a speaker. The gateway then reads every reply aloud for that chat, both typed and spoken input. /voice off in the device's chat, or speak_replies: false, turns it off.
Interpreter mode
In interpreter mode a gadget translates between two languages instead of answering (how to use it). plugin/interpret.py holds the parts that do not depend on Hermes.
No agent turn. The person holding the gadget may be a stranger, so the adapter makes no MessageEvent for what they say. It runs the three steps itself, with Hermes's own services:
- Speech-to-text.
tools.transcription_tools.transcribe_audio, the function the gateway uses for voice messages, with the recording's language fixed through thepre_transcriptionhook, as forstt_language. The recording is deleted when it returns. - Translation.
ctx.llm.complete(...), Hermes's completion API for plugins (agent/plugin_llm.py), on the model the profile is configured with. The request is two messages: a system prompt that asks for the translation and nothing else, and the text with its direction. The API takes messages only; it has no parameter for tools. - Speech.
tools.tts_tool.text_to_speech_tool, as for a message sent outside a turn, played to the device.
The device sees an ordinary turn: turn.start, transcript, reply, reply audio, turn.end.
What this enforces, by construction and not by a prompt:
- The utterance cannot trigger a tool call or an approval prompt. No agent runs, and the model request carries no tool schemas. The model's reply is used as text to show and speak, nothing else.
- No Hermes session exists for a translation, so nothing is written to the session store, no session title is generated, no memory review runs, no memory provider sees a turn, and
session_searchhas nothing to find. - The device's conversation is never in a translation request, and no translation is ever in the conversation.
What it does not enforce: the words go to the configured STT, model and TTS providers, and Hermes writes its usual audit line for a plugin completion (plugin, provider, model, token count; no text) to the gateway log. Hermes retries a failed completion, and where the configuration lists fallback providers for auxiliary calls, may send it to one of them. A translation that fails logs the error text.
Failing closed. A gadget in the mode may be in someone else's hands, so doubt about whether the mode is on is resolved toward "not an agent turn":
- The mode is kept in
interpret.jsonin the plugin's data folder and read when the gateway starts. A restart does not end it. A mode that cannot be written there does not start:interpret_startfails with the reason and the device stays as it was, because a restart would end an unsaved mode without anyone noticing. - The device keeps the mode through a lost connection, and marks what is typed or recorded in it (
interpret: trueintextandaudio.start), a recording it sends again after reconnecting included. The hub hands a marked message toon_interpret_textoron_interpret_utterance, never toon_textoron_utterance. The adapter translates it while the mode is on, and otherwise drops it and tells the device the mode is off. A mode that expires on a message drops that message too. - When a device connects, the adapter states the mode in so many words, on or off, before anything else.
- While the mode is on, the adapter shows no Hermes question on the device (
prompt) and ignores an answer from it; the question stays queued, and is shown when the mode ends unless it was answered elsewhere. A question on screen when the mode starts is withdrawn from the device. - The agent cannot ask the gadget a question either.
gadget_askanswers with an error while the mode is on, and a choice that is on the screen when the mode starts is withdrawn and its tool call fails: a tap by whoever holds the gadget is never returned to the agent as the user's answer. The hub enforces this where the choice is made and where an answer arrives, on the gateway's loop. It asks its delegate (HubDelegate.interprets) before it creates a choice, so a mode that starts after the tool looked still refuses it, and it drops achoice.replyorprompt.replyfrom a device that interprets. - The gadget's profile cannot be switched from its settings while the mode is on: the request is answered
busy. - A device event with
notifystarts no turn while the mode is on. - A message sent to the device outside a turn the owner started (
send_message, cron delivery) is held, up to 20, and delivered when the mode ends. A message leaves that queue only when it was sent: after a send that fails, it and the later ones wait for the next time the device connects or leaves the mode. Unpairing the device discards them, also when the mode expires or a send fails at the same moment. send_voice,play_ttsand images are refused with an error, an image also when the mode started while it was being converted.- Only a translation of the device's current mode is played. Hermes's own speech, such as a message spoken outside a turn or a spoken
clarifyquestion, is checked again right before the audio starts: speech that was queued or still being synthesized when the mode started is dropped. Its text was shown before the mode started. Audio that is already playing when the mode starts is not stopped. - Holding CANCEL sends
interpret.stop, neversession.new, so it cannot reset the owner's conversation.
One turn is let through: if the agent turned the mode on with gadget_interpret in the gadget's own chat, the rest of that turn, its reply included, still reaches the device. Another agent turn that is running on the gadget when the mode starts, for example when the mode is started from another chat or with hermes gadget interpret, reaches it no more: its status phrases, streamed text and streamed speech are dropped, and its finished reply is held like a pushed message.
When the mode starts, the adapter also stops what the gadget is playing: a gadget_play playback and speech still being sent. A choice the agent asked is withdrawn when the device is unpaired, as when the mode starts.
What is kept: nothing of a translation is written to a session or to memory. The recording is written to a temporary file that the same worker deletes when speech-to-text returns, also when the turn was cancelled meanwhile; a speech file whose synthesis outlives a cancelled turn is deleted when it is done. While the mode is on, what was heard and replied is left out of status.json, which the Desktop page mirrors.
Bounds. Speech-to-text has 60 seconds. The completion is asked for at most 512 tokens with a 20 second timeout per attempt, and 45 seconds for all attempts together; the translation is cut to 800 characters.
Entering and leaving. The adapter keeps the mode per device: the two languages and which one speaks next.
hermes gadget interpret <device> <language> <language>posts a request to the gateway through the same file queue ashermes gadget diag.offin place of the languages leaves the mode, also for a device that is not connected.- The
gadget_interpretagent tool does the same from a conversation. - On the device, holding CANCEL sends
interpret.stop. Firmware that does not know that message sendssession.new; while the device interprets, the adapter leaves the mode on it and sends no/new. A translation under way is stopped. - The mode ends after 30 minutes without a translation or a switch of speaker, and when the device is unpaired. The device is told.
- The device is always told the mode the adapter holds at that moment. When a stop and a new start overlap, for example CANCEL held while the mode is started again with other languages, the device ends in the new mode and gets no "Interpreter off".
- The mode is refused when it is started on a Hermes without
ctx.llmor without the speech functions above, and when it cannot be saved tointerpret.json.
The adapter tells the device with an interpret message.
The language of each recording. Hermes's speech-to-text takes at most one language per recording and does not report which language it heard. Hermes's default configuration fixes English (stt.language: en), and a pre_transcription hook can replace a language but not clear one, so a plugin cannot ask for detection either. The mode therefore does not detect who is speaking: the speakers take turns. The adapter notes each recording with the language whose turn it is, and the hook hands that language to the STT provider. The speaker switches when the translation is sent to the device, before it is spoken, so someone who answers during the speech is recorded in their own language. A turn that fails or is cancelled switches nothing. The device's interpret.flip message, a short CANCEL press, switches the speaker by hand.
The reply. The adapter folds the translation to the device's charset as usual. When that drops letters or digits because the font lacks them, the device is told: a note under what is left, or, when a third or more is gone, a line in place of the text. That line says "spoken only" only when the translation is about to be spoken: the device has a speaker and voice is on for its chat. The audio is made from the model's text and is not affected.
Limits:
- The gateway's profile translates. The adapter calls
ctx.llmand the speech functions from the gateway process, as the profile that runs the gadget platform, whichever profile the device selected or a route sends it to. This is verified on a gateway that serves one profile; on a multiplexed gateway it is untested which profile's settings those calls read. - Each utterance is translated alone. The model gets no earlier sentences, so it cannot use them to resolve a pronoun or a name.
- One voice. Hermes has one text-to-speech voice per profile, and the plugin cannot choose a voice per language. Speech is made as a whole file, not streamed.
- A wrong turn is not detected. A recording transcribed in the other person's language comes back as a wrong translation.
- Firmware that does not mark. A device without
caps.interpretsends nothing marked, so for it only the gateway's own record of the mode keeps a message from the agent. - 54 languages.
interpret.LANGUAGESlists the codes the mode accepts; your STT, model and TTS must each support the pair.
Pairing
The SDK reuses Hermes's DM pairing instead of inventing its own:
- When an unknown device connects, the adapter dispatches a harmless
/statusmessage from it. - Hermes's unauthorized-sender path answers with a pairing code. The adapter recognizes the
hermes pairing approve <platform> <CODE>command in that reply and sends the device apairingframe. - The device shows the code. The owner runs
hermes gadget pairon the Hermes host, which finds the waiting device and approves its code after asking, or runs thehermes pairing approvecommand the device shows. - The adapter polls the runner's authorization check every 2 s, so approval reaches the device as
pairedwithout a reconnect. Revocation (hermes pairing revoke gadget <id>) reaches it asunpaired.
Home channel. Hermes opens each new session on a platform without a home channel with a "type /sethome" notice, which a device without a keyboard can't act on. So when a device is approved and the gadget platform has no home channel (platforms.gadget.home_channel or GADGET_HOME_CHANNEL), the adapter makes that device the home channel, as /sethome would, and saves it to config.yaml. Cron results and messages sent to gadget then go to that device. Run /sethome from another device to move it, or set auto_home: false to turn this off.
The pairing code authorizes the chat in Hermes. The device key, enrolled on first contact and proven by HMAC afterwards, keeps another device from impersonating an approved one.
Until Hermes approves a device, its record is pending: at most 16 devices may wait at once, at most 4 from one network address, and a pending record expires an hour after the device's last contact (it simply re-enrolls on its next connection). The hub refuses further strangers with an error of code busy. hermes gadget devices marks pending devices with "waiting to pair". hermes gadget pair --yes approves without asking only when exactly one device is waiting; with several, name one: hermes gadget pair --yes <device>.
Confirmations
Hermes asks before destructive commands (/new, /undo), costly model switches and dangerous shell commands. Chat platforms render those as buttons; the gadget renders them as a yes/no prompt (see protocol.md):
- Holding CANCEL sends
session.new, which the adapter turns into/new. Hermes then asks "Confirm /new". The hold already was a deliberate confirmation, so when the confirmation fornewarrives within 15 s of the device's own request, the adapter resolves it withonceand forwards the reset reply. A/newtyped or spoken any other way still asks. - Everything else goes to the device as a question. The adapter keeps only the header and detail of Hermes's message, dropping the choice list and typed-reply instructions meant for chat apps.
- Dangerous commands are shown exactly as they will run, never reshaped: no Markdown stripping, folding or dropped characters. Only a command in printable ASCII is shown; the device collapses runs of spaces and keeps one line of a question on its smallest screens, so a command is shown only when it fits one line of the device's
text_cols, less two columns for the..the device writes when it cuts the description below. Any other command (longer, multi-line, with a tab, double space or non-ASCII character, or for a device that reports notext_cols) is shown as "Can't show this command": CANCEL still denies it, and TALK does nothing, so it must be approved on the host. - Paired devices only: a device that is unpaired, or that reconnects unpaired, loses its open questions, and an answer from an unpaired session is ignored.
- One at a time: questions queue per device in arrival order, which matches Hermes's first-in, first-out approval queue. A question asked while the device was offline is shown when it reconnects.
- Elsewhere: answering with
/approveor/denyfrom another chat still works; the device's copy then reports "That question has expired" if pressed.
Agent tools and Hermes tool search
Hermes keeps the core tool schema narrow. Plugin tools are therefore deferred behind its tool-search bridge (tool_search / tool_describe / tool_call), and their names and descriptions appear in the bridge's manifest. A model invokes gadget_action through tool_call, and the end-to-end test exercises exactly that. The per-device context names the available actions so the model knows they exist.
The eight tools:
| Tool | What it does |
|---|---|
gadget_devices |
Lists connected gadgets with their screen, audio, actions and sensor readings |
gadget_display |
Shows a card: title and text, and optionally items (a list), progress (0 to 100) or qr (text to show as a QR code). A card sent again with the same id replaces the one on screen in place, which is how a progress bar moves |
gadget_ask |
Shows question with 2 to 6 options, waits up to seconds (60 by default, 300 at most) and returns the option the user picked, as the agent wrote it, with its index; cancelled: true when the user cancelled; an error when nobody answered |
gadget_action |
Runs an action the device declared |
gadget_speak |
Says text out loud with Hermes's text-to-speech (see Speaking from any chat) |
gadget_play |
Plays a sound file or an audio stream on the device's speaker, and stops it (see Playing audio) |
gadget_widget |
Sets, lists and clears home-screen widgets (see Widgets from the agent) |
gadget_interpret |
Turns a gadget into an interpreter between two languages, and back (see Interpreter mode) |
gadget_display and gadget_ask shape their text for the device: Markdown is stripped, characters are folded to its charset, and each list row or answer is one line of at most text_cols less 2 characters, the room left after its bullet. A QR code's text is sent unchanged.
The agent learns when the user saw something else than it wrote. gadget_ask blocks the turn, so its screen is all the user has to go on: a question longer than the device shows (caps.choice.title_cols by title_lines) is cut to fit and returned as question_shown, options that were cut come back as options_shown, and note says so. gadget_display returns items_shown when the device keeps fewer rows than the list has.
What a device cannot draw still reaches the user. A gadget without caps.card (older firmware, the Linux client) gets a list, a progress value and a code's text as a plain card, and the tool's result says so in note. A gadget without caps.choice gets the options as a numbered card; gadget_ask then fails with shown_as_card: true and tells the agent to ask for the answer by voice. See structured cards and choices.
gadget_ask is separate from confirmations. A dangerous command is still approved through Hermes's own yes/no question, with the rules above; an agent cannot stand in for that approval with a choice.
Speaking from any chat
gadget_speak says text on a gadget's speaker from any session, for example "tell the kitchen dinner is ready" in a Telegram chat. Markdown is removed. The text goes through the same per-device queue as a message sent outside a turn (see Scheduled delivery): it is spoken after a turn in progress and after anything the gadget is already saying, one message at a time, and never cuts either off. Like any speech, it stops a playback from gadget_play. The tool returns as soon as the text is queued, with status saying whether it starts now or waits, so the agent is not held up behind other speech. A cron job is not needed, and would wait for the scheduler's next tick.
The tool refuses a device without a speaker, one waiting for pairing, one in interpreter mode, and one where voice is off (/voice off in its chat, or speak_replies: false). Text-to-speech runs once the message's turn comes, so a text-to-speech failure, or a device that stays busy longer than the queue waits, is not returned: the message goes to speech_overflow_target, or is logged without one (see Scheduled delivery). Unlike gadget_play, speech is not stopped when Hermes speaks, and it does not show a Playing notice: it is a message, not a playback. Starting a recording on the gadget stops it, as it stops any audio the gateway sends.
Playing audio
gadget_play plays something other than speech on a gadget's speaker: a sound file on the Hermes computer, or the http:// or https:// URL of an audio file, a podcast episode or an internet radio stream. Ask for it in words ("play the BBC World Service on the kitchen gadget", "stop the music").
| Argument | Meaning |
|---|---|
source |
A file path on the Hermes computer, or an http(s) URL of the audio itself |
stop |
true stops what gadget_play is playing |
volume |
1 to 100: the level of this playback, as a percentage of the source's own. The speaker's volume is still the device's speaker.volume action |
title |
A short name for the notice on the screen; the file name or the host otherwise |
device |
Which gadget, as for the other tools |
How it plays:
- On the Hermes computer. The plugin decodes the source and sends it through the same paced stream as a spoken reply (Audio to the device), so it needs no firmware support beyond a speaker. Plain PCM WAV is read in Python. Every other format, and every URL, needs
ffmpegon the Hermes computer. - A chunk at a time. The source is decoded about a second ahead of what the device has been sent, and a URL is downloaded only as fast as it plays. A long file or an endless stream is never held in memory.
- After the reply. The tool answers once the source has been opened and its first audio decoded, so a URL that does not answer or a file that is not audio is the tool's error. A source that gives no audio within 15 seconds of the tool call is given up on, and the download and
ffmpegstarted for it are ended. The audio starts when the gadget has finished the turn and its spoken reply; a gadget that stays busy for two minutes never starts it. The screen showsPlaying <title>as a notice. - Until something else needs the speaker. Playback ends at the end of the source, at
play_max_minutes(60 by default, counted both as audio played and as time on the clock, so a stream that arrives slowly ends on time too), when a stream sends no audio for 20 seconds, onstop: true, when the device disconnects, on CANCEL on the device, when you start talking to the gadget, or when Hermes speaks: a reply, or a message sent outside a turn such as a reminder, which does not wait for the music to end. It does not resume afterwards. A playback that is still waiting for the reply to end is ended the same way by CANCEL, by a new recording and bystop: true, and never starts. After the source has ended,stop: truestill ends the audio that has not been sent to the device. What the device has already been sent, half a second of audio and up to 3 seconds on a device with a large buffer, plays out: the firmware ignoresaudio.abortfor a stream whoseaudio.endit has received. A secondgadget_playreplaces the first.
What it may open:
- Local files only inside Hermes's audio cache (where its text-to-speech files and received audio are kept) and the folders listed in
platforms.gadget.extra.play_dirs. The path is resolved first, so a symbolic link or..that leads elsewhere is refused, with the same answer whether or not the path exists. A playlist file (.m3u,.m3u8,.pls, or a file that begins like one) is refused, because it names other files. Withoutplay_dirs, nothing else on the host can be played. - URLs only with the
httporhttpsscheme, without a user name or password. They are fetched with Hermes's own guarded HTTP client under Hermes's URL policy: private, loopback and link-local addresses are refused when the name is resolved, when the connection is made and on every redirect, unless Hermes'ssecurity.allow_private_urlsistrue; cloud metadata addresses are refused always. When Hermes runs behind a proxy (HTTP_PROXY,HTTPS_PROXYorALL_PROXYis set), it leaves name resolution to the proxy: the check when the connection is made does not apply, and a name the Hermes computer cannot resolve itself is passed to the proxy unchecked. Restrict what the proxy may reach in that case. To play from a media server on your own network, set that Hermes option. One URL sends at most 512 MB. ffmpeggets no access of its own. A URL's data is piped into it and it may open nothing else (-protocol_whitelist pipe); for a local file it may open files only. In both cases it reads audio containers only (-format_whitelist: MP3, AAC, Ogg, FLAC, WAV, MP4/M4A, Matroska/WebM, MPEG-TS, ASF/WMA, AIFF, AU, CAF, AC-3, AMR, APE, TTA, WavPack), not the playlist formats that open other files or URLs by name, so a playlist or container cannot make the Hermes computer read or fetch anything else.- Playlists are not followed. For an
.m3uor.plsURL the tool returns up to three of the stream URLs the playlist lists, and the agent picks one. That text comes from the remote server: only plainhttp(s)URLs of at most 200 characters are passed on. HLS streams (.m3u8) are not supported. - Errors say little. What
ffmpegprints about data it could not decode, and the details of a failed download, go to the gateway log, not to the agent. - Only a tool call fetches. Nothing a device sends, and no message text, makes the plugin open a file or a URL.
Widgets from the agent
gadget_widget sets, lists and clears home-screen widgets. It writes the same file as hermes gadget widget (dashboard.json in the plugin's data folder), so a widget the agent sets and one a cron job sets are the same thing, and either can replace or clear the other's.
| Argument | Meaning |
|---|---|
action |
set (the default), clear or list |
name |
The widget's name. Setting it again replaces the widget. clear without a name removes every widget of that scope |
kind |
value, progress, list or headline |
label, value, unit, percent, items, title, text, state |
The widget's fields, as in the protocol |
ttl_s |
Seconds until the widget disappears unless it is set again |
device |
A gadget's name or id, connected or not. Without it the widget is for every gadget |
- The tool does not take the screen and does not wait for a device: the gateway sends the changed dashboard to connected gadgets at once, and to the others when they connect or their screens light up.
- The limits are the CLI's: eight widgets for a gadget, counting the shared ones with its own, four list items, and text cut to the device's line. A widget for every gadget is refused when it would give any gadget a ninth, and the error names that gadget, so a shared widget never displaces a gadget's own.
ttl_sis a positive number of seconds. - The tool answers with an error, and stores nothing, for a widget no device could draw: a missing value, a percent that is not a number, more than four items. It also answers with an error when the file cannot be read or written.
- Every change is one locked read-modify-write of
dashboard.json: the CLI, a cron job, the tool and the gateway can change tiles and widgets at the same time without losing each other's changes. A change waits up to 10 seconds for another to finish. The lock is the operating system's lock ondashboard.json.lock, so it ends with the process that holds it: a process that died blocks nobody, and a lock is never taken over because its file is old. - The gateway picks each device's layout from its
caps.dashboard.v. Older firmware, and boards that cannot page, get each widget as a tile on the sessions list. - With
dashboard: falsethe tool reports that the dashboard is off.
Making replies fast
A voice turn is speech-to-text, the model turn, then text-to-speech. Most of the wait is usually the model, and the settings that matter live in Hermes:
| Setting | Effect |
|---|---|
/reasoning low (or none), typed on the device |
Session-only reasoning override for the gadget's chat. A deep-reasoning default (agent.reasoning_effort: high) can spend 10–20 s thinking about a one-line answer. Other chats keep their setting |
streaming.enabled: true |
Streams reply text to messaging platforms as it is generated, so words appear on the device within a second or two. Hermes's per-platform display.platforms.<name>.streaming can only narrow this global switch, so on a shared gateway set the global on and switch the other platforms off. The Desktop app streams on its own and is not affected |
| A TTS provider Hermes can stream | Speech starts after the first sentence instead of after the whole reply. While whole-file TTS synthesizes, the reply text waits for it, so a working streaming provider also shows the text sooner. Check the gateway log for streaming TTS clause failed; it usually means a bad API key |
stt.provider |
Cloud STT takes about 2–3 s for a short clip. Local faster-whisper can be quicker on a fast CPU |
Where the SDK leans on behavior that is not a formal API
plugin/compat.py lists every gateway module the adapter imports and every private BasePlatformAdapter member it overrides or calls, with the Hermes commit CI tests against. The gateway runs that check before it creates the adapter, so a Hermes that has changed one of them fails with a message naming it, instead of a traceback from inside the gateway. A test keeps the list complete.
These work on current Hermes and are covered by tests/test_adapter_hermes.py and tests/test_gateway_e2e.py, but they are conventions rather than documented contracts:
- Spoken replies by default. The adapter overrides
BasePlatformAdapter._should_auto_tts_for_chat, a private method the runner consults. Hermes's own docs endorse overriding private hooks such as_keep_typingfor platform UX, but this one is not listed. - Pairing trigger and code capture. Starting pairing takes a synthetic message, and the code is extracted from the human-readable reply by matching the approve command.
- Approval detection polls
_is_sender_authorizedevery 2 s, because there is no "pairing approved" event. - Transcript echo detection matches the echo format: microphone emoji plus quoted text. If the format changes, the transcript is shown as an ordinary reply; nothing breaks.
- Self-confirming
/newreadstools.slash_confirm.get_pending()to check that the pending confirmation is for thenewcommand. It relies on the runner registering a confirmation before it callssend_slash_confirm, which Hermes does on purpose so that fast button presses can't race it. If that changes, the device simply asks. - Question text is cut down from Hermes's chat-formatted confirmation by dropping the bullet-list and italic paragraphs. A different layout only means a longer question.
- Dashboard session state comes from plugin hooks, which are a public API:
pre_llm_call(busy),on_session_end(idle),pre_approval_requestandpost_approval_response(waiting), andon_session_finalizeandon_session_reset(gone). It relies onsession_idin their keyword arguments. The approval hooks carry it only when Hermes binds the approval to a session; an approval without one doesn't show as waiting. Hermes loads a copy of the plugin per profile, and a profile's hooks see only that profile's sessions. A device that switches profiles also gets the sessions of each offered profile that has the plugin enabled (profile switching). hermes gadget bot-chatsends/resume --all Bot Chatas the device and reads the profile'sstate.db(thesessionsrow titledBot Chat) to see that it moved. It relies on Hermes's documented Bot Mode rule that the canonical session is the one titled exactlyBot Chat, and on/resume --allbeing allowed to admins listed inallow_admin_from.
Suggested upstream changes (optional; nothing here blocks the SDK)
Each would replace a workaround above with a small, generic hook that every platform plugin could use. None of them is specific to gadgets. The first four are open pull requests on Hermes Agent; until they merge, the SDK keeps the workarounds.
Pairing API for adapters: NousResearch/hermes-agent#132448, open.
- What:
BasePlatformAdapter.request_pairing(source) -> PairingOffer | Nonereturns the code the DM path would send (code,command,expires_in), behind the same gates and limits, without sending anything to the chat.on_pairing_changed(user_id, approved)is called once for each approval and revocation, from one watcher in the runner.
- Why: devices, kiosks and any screen-first platform could show codes natively, without synthetic messages, text parsing or polling.
- SDK change afterwards:
on_readycallsrequest_pairing, and_watch_pairingis deleted.
- What:
Voice-first platforms: NousResearch/hermes-agent#132441, open.
- What: two adapter class attributes.
speaks_replies_by_default = Truespeaks replies withoutvoice.auto_tts, and/voice offstill silences a chat.supports_voice_replies = Falsekeeps replies text, replacing the name checks for A2A. - Why: voice-native platforms (gadgets, phone bridges) want spoken replies by default without overriding a private method.
- SDK change afterwards: the
_should_auto_tts_for_chatoverride is removed, andspeak_repliesmaps onto the attribute.
- What: two adapter class attributes.
Transcript echoes for adapters: NousResearch/hermes-agent#132435, open.
- What:
BasePlatformAdapter.send_transcript_echo(chat_id, transcript, metadata=None). By default it makes the samesend()call as today; an adapter that overrides it gets the raw transcript. - Why: adapters could render transcripts distinctly without matching localized text.
- SDK change afterwards: the adapter overrides it, and the echo-text matching is deleted.
- What:
Direct toolsets for a platform's own sessions: NousResearch/hermes-agent#132449, open.
- What:
PlatformEntry.direct_toolsets. A toolset listed there is direct only in sessions on that platform. - Why: a platform-specific tool (here, controlling the device the user is holding) is part of that surface, and a search round trip costs latency on voice turns. Sessions on other platforms keep deferring it, so the core schema stays narrow.
- SDK change afterwards:
register_platform(..., direct_toolsets=("gadget",)).
- What:
Per-platform reasoning default.
- What: a
platforms.<name>.reasoning_effort, resolved after the session override and before the per-model and global settings. - Why: voice devices want fast, shallow turns while desktop chats keep deep reasoning. Today that needs a
/reasoningper session. - Upstream status (October 2026): an open pull request, NousResearch/hermes-agent#65576, adds
reasoning_efforttochannel_overrides, which is per chat (per device). A platform-wide default would be a small follow-up to it rather than a separate change. - SDK change afterwards: document a
platforms.gadget.reasoning_effort: lowdefault.
- What: a
Running against a profile or a multiplexed gateway
A multiplexed gateway runs every profile in one process. Enable the gadget platform in the default profile only: that one adapter serves every profile. When you turn profile switching on, each device chooses which profile answers it from its settings (protocol). A gateway that is not multiplexed offers no profiles, and its devices talk to the profile it was started with.
- Turn it on. Switching is off by default, and devices talk to the default profile as before. Set
platforms.gadget.extra.profilesin the default profile'sconfig.yaml. A list, such asprofiles: [work, home], offers those profiles and the default one.profiles: alloffers every profile the gateway serves. A device shows at most 32 of them: the default profile and the next 31. Each one is named as the Hermes interface names it: its display name, else its Desktop Bots title (ui_meta['hermes-bots'].titleinprofile.yaml), else its id with a capital first letter. Parked profiles and those withgateway.standalone: trueare never offered. The plugin reads the key only from the default profile's config, so a profile cannot add itself. A profile that is parked or deleted while a device uses it, or that you take off the list, sends the device back to the default profile at the next pairing check. - Native session admission. On Hermes versions with session-authority admission, device-selected routing also needs the gateway's
validate_native_profile_routeadapter hook (NousResearch/hermes-agent#147690, which supersedes #143621). That pull request is still open, so no Hermes release has the hook yet, and neither doesmain. From v0.22.5 on, switching is accepted, then every message for the new profile fails withprofile_mismatch, and the default profile keeps working (#232). Until the hook lands, switching to a profile other than the default one is unavailable on those versions. The plugin rechecks the offered profile, the device's Default-owner approval and its owner binding when input is admitted or replayed. A queued input keeps its original profile after a later device switch, but revocation or removal from the offered profiles still refuses it. A successfulprofile.selectfollowed byprofile_mismatchinsession_ingress_context._bindingusually means the Hermes version lacks that hook, and updating only the firmware or plugin is not enough. The hook also refuses on purpose when the profile is no longer offered, the default profile no longer trusts the device, or the device isn't owner-bound, so check those first. Don't work around it by disabling a session-authority check or addinggadgetto_PREFIX_PRODUCERSinsession_ingress_context.py: that accepts any profile a gadget names, without the plugin's checks. - Approve the device in each profile. A device authorized in the default profile (an approved pairing code,
GADGET_ALLOWED_USERS, or an allow-all setting) may switch to any offered profile, but each profile keeps its own approvals: the plugin writes nothing into another profile. The first time the device talks to a profile that doesn't know it, that profile's own pairing flow offers a code, and the device shows the command to run, such ashermes -p work pairing approve gadget <code>. Approve it, then ask again. - Revoke. Remove the device's authorization in the default profile:
hermes pairing revoke gadget <id>, and take it out ofGADGET_ALLOWED_USERSor turn allow-all off if that is how it was let in. Within a few seconds it is moved back to the default profile, its pending questions are dropped, and it cannot switch again. Its approvals in other profiles belong to those profiles; remove them there withhermes -p <profile> pairing revoke gadget <id>. When a device first switches away from the default profile, the plugin marks it as bound to the default profile (an empty file inplugin-data/gadget/owner-bound/). A marked device is checked against the default profile before every message for another profile, even if it later connects with firmware that cannot switch. Nothing deletes the mark,hermes gadget forgetincluded.hermes gadget forget <id>only resets the device's enrolled key so it can enroll again; it is not a revocation. - Before you enable a gadget listener in another profile, note that the approvals you gave a device there let it in through that listener too, from firmware that cannot switch as well. The mark lives in the default profile's plugin data, so the other listener doesn't see it. Revoke those approvals first if that is not what you want.
- Keep the plugin enabled in every profile a gadget may use, with only its gadget listener off (see the first item under One profile per gadget). The profile needs the plugin for the gadget tools, and for its sessions to appear on the dashboard of a device that switches.
- Switching. A switch is refused, and can be tried again, while the device has a turn running or a question pending. A reply that was already on its way from the old profile may still appear. Each profile keeps its own conversation, memory and personality, so switching does not carry the chat over.
Device keys and pending pairing codes live in the plugin data directory. It is resolved with plugin_data_dir("gadget") when the adapter connects, so it follows whichever Hermes home the adapter is started under. The gateway re-reads devices.json before each lookup or change, so hermes gadget forget and hermes gadget pair from another terminal take effect without a restart.
The agent tools look up the adapter for the session's profile (HERMES_SESSION_PROFILE), and fall back to the only adapter running. Hermes imports the plugin once per profile scope, so the registry of running adapters is kept once per process rather than in each copy of the module.
One profile per gadget
A multiplexed gateway can give each gadget its own profile (model, memory, sessions and personality) while all of them connect to one listener. Keep the gadget platform on the profile that owns the port, usually default, and route each device by its id:
gateway:
multiplex_profiles: true
profile_routes:
- name: gadget-kitchen
platform: gadget
chat_id: "hg-0123456789abcdef"
profile: kitchen
Then, in each routed profile:
- Turn its own listener off with
hermes -p kitchen config set platforms.gadget.enabled false. A profile cloned fromdefaultkeeps the plugin enabled, and otherwise starts a second listener that retries the taken port. Leave the plugin enabled: the profile needs its tools. - Approve the device again with
hermes -p kitchen pairing approve gadget <code>. Pairing is checked in the routed profile, so the device shows a pairing code after the route takes effect. - Update its copy of the plugin too. Cloning a profile copies
~/.hermes/plugins/gadgetinto~/.hermes/profiles/kitchen/plugins/gadget, and the routed profile runs that copy: after updating the plugin, copy it into every cloned profile and restart the gateway, or the profile keeps the old code. A copy older than this release cannot see the default profile's devices, and the agent reports thatgadget_actionis unavailable.
Hermes's tool search can keep plugin tools out of the model's tool list until the model searches for them. A model that answers without searching then says it can't change the volume or brightness. If the gadget tools are listed as available but the agent never calls them, turn tool search off in that profile with hermes -p kitchen config set tools.tool_search.enabled off and start a new session, so the earlier refusals aren't in its history.
Tried with three devices routed to three profiles on one gateway: the turns from the two devices used so far each ran in their own profile's session.
Profile switching and profile routes
gateway.profile_routes and profiles do different jobs. A route pins one device to one profile, chosen on the host. The plugin offers a routed device no profiles, so its settings have no Hermes profile item, and it keeps the routed profile's pairing. profiles lets every device without a route pick its own profile, from the ones you list. To let a routed device switch, remove its route; it then needs approval in the default profile. The reverse doesn't work: once a device has switched, it answers to the default profile for good, and a route added for it later makes the plugin refuse it. The device shows "Refused by a route" and its device ID in place of a pairing code, its status has "unpaired_reason": "routed", and the gateway log says so too. Hermes reads the routed profile's approval for a routed device, so the plugin cannot check the default profile's until Hermes offers a public per-profile check (NousResearch/hermes-agent#143892). Remove the route and restart the gateway to let it in again; if the default profile no longer approves it either, it goes back to pairing.
A gadget that talks to the other Bots
With Hermes Bot Mode, a profile's canonical Bot Chat session gets the message_agent tool: the agent there can message the other Bots and hears their answers when they come. Hermes Desktop shows those messages in each Bot's chat. A gadget's conversation is a session of its own, so by default it has no message_agent. hermes gadget bot-chat moves a profile's Bot Chat onto a gadget, so that profile leads from the device: ask it "ask Nick and Nora what they think" and it messages them, then says what they answered. The conversation is the same one Hermes Desktop shows for that Bot, from either side.
Bots. Make the profiles Bots in Hermes Desktop's Bots pane, which creates each one's Bot Chat. Without Desktop, run
hermes -p <profile> chat -c "Bot Chat" --create-if-missingfor each, and putui_meta: {hermes-bots: {}}in one profile'sprofile.yaml(Bot Mode).Make the gadget an admin of the gadget platform. Hermes lets a chat take over a session from another surface only for an admin (
/resume --all). In the default profile'sconfig.yaml:platforms: gadget: extra: allow_admin_from: [hg-0a0eab5baf3b217d] # the leading gadget, from `hermes gadget devices` user_allowed_commands: [new, stop] # other gadgets keep "new conversation" and "stop"Once
allow_admin_fromis set, gadgets not on it may run only/help,/whoamiand the commands inuser_allowed_commands. Gadgets send/newthemselves when CANCEL is held, so keepnewthere. Restart the gateway.Run it:
hermes gadget bot-chat <device> [profile], withdefaultwhen no profile is named. The gateway switches the device to that profile if it isn't there, sends/resume --all Bot Chatfrom it, and the command waits until the profile's Bot Chat is on the device's chat. It works over the network: no USB console. A profile other than the default one must be offered to the gadget (profiles), and on Hermes with native session admission it also needs thevalidate_native_profile_routehook, which no Hermes release has yet (see Native session admission).
The command refuses before it asks the gateway when the profile has no Bot Chat, the gadget is not on allow_admin_from, or the profile isn't offered, and it prints what to change. The gateway serves the request once the device is online, paired and idle (no turn or question pending), and keeps it for 10 minutes; the command itself waits 60 s (--timeout), so the terminal can give up while the request still runs. Running the command again doesn't check on it: it replaces the request with a new attempt and its own 10 minutes. It reports a failure when the device reconnects or leaves the profile while being switched, or when the default profile no longer trusts it. A profile that hasn't approved the gadget yet answers the resume with a pairing code, which the gadget shows, and the command gives up when its --timeout runs out: find the code with hermes -p <profile> pairing list, approve it with hermes -p <profile> pairing approve gadget <code>, and run the command again. A newer run for the same device replaces an older one's request. Run it again for each profile that should lead on that gadget; a profile's Bot Chat follows the last chat that took it, so opening it from Hermes Desktop afterward continues the same conversation.
A Bot added later needs nothing on the gadget: message_agent checks its target against the Bots Hermes has at the time, and the Bot Chat's teammate list is rebuilt when the Bots change.