The simulator

New here? Follow Try the simulator for installation and your first conversation. This page covers controls, board profiles, and scripting.

hermes-gadget sim runs the production device core (firmware/core) as a shared library inside a Python host.

Real code, same as on the ESP32:

• protocol, authentication and enrollment;
• reconnect and backoff, heartbeat, pairing UX;
• push-to-talk and VAD, playback control;
• the UI renderer, device actions, telemetry, the serial console.

Simulated, replacing the drivers:

• Wi-Fi: a checkbox;
• the WebSocket: Python websockets;
• the display: a Tk window;
• microphone and speaker: WAV files, or PC audio with --live-audio;
• NVS: a JSON file;
• buttons: the keyboard.

A bug reproduced in the simulator is therefore a bug in the firmware.

Running

The main window shows the device and the current conversation. Developer tools replaces the conversation panel with sensors, device actions, the serial console, and logs. Choose Back to conversation to return.

Settings changes the gateway address, board profile, microphone, and speaker. Audio uses the system's default devices; change those in your operating system's sound settings. A board change restarts the simulated device and keeps its stored identity. Large boards use a smaller zoom to fit the display, including half size on short screens.

Typed messages and voice transcripts appear beside replies as they stream. Copy reply copies the latest reply. Play last audio replays the latest recorded reply through the computer's default output. Conversation text stays in memory, limited to the last 100 messages, and clears when you start a new session. Audio files remain in the state directory described below.

hermes-gadget build-sim                     # once, and after changing firmware/core
hermes-gadget sim --url ws://127.0.0.1:8765/gadget [--board sim-320x240] [--name "Desk"] [--live-audio]

A first session, start to finish:

1. Start Hermes with the gadget plugin (or hermes-gadget devserver --pairing for a stand-in that needs no Hermes).
2. Run hermes-gadget sim --url ws://127.0.0.1:8765/gadget --live-audio --board sim-466x466-round.
3. The screen shows a pairing code. Approve it on the Hermes host: hermes gadget pair (or hermes pairing approve gadget <CODE>).
4. Hold Space, ask something, let go. Hermes answers on screen and through your speakers.
5. Hold Esc for 2 s to start a fresh conversation.

The simulated device registers two demo actions the agent can use: led.set (a virtual LED in the window) and buzzer.beep.

The device settings menu runs in the native firmware core. On touch profiles, hold the device title bar to open it. You can also enter settings in the serial console. The desktop Settings dialog configures the simulator host; the device menu changes saved volume, brightness, and talk mode.

State lives in ~/.hermes-gadget/sim/<name>/ (override with --state-dir):

• nvs.json: device key and settings. Delete it to look like a brand-new device.
• audio/: every reply the speaker played, as WAV.
• screenshots/: screenshots saved with Ctrl+S.
• update.bin: the last firmware image installed over the air.

Like a board, the simulated device takes firmware updates (hermes gadget update, or update <path> in the dev server). It checks the image the way the core does and keeps only ESP32 app images. Then it "restarts": it drops the connection and connects again, still running the simulator.

Boards

Add a profile to BOARDS in python/hermes_gadget/sim/runner.py to mirror new hardware.

Headless and scripted runs

hermes-gadget sim --headless --url ws://127.0.0.1:8765/gadget --script examples/scripts/smoke.txt

Script commands, one per line:

Screens are boot, offline, connecting, pairing, ready, listening, thinking, responding, card, image, prompt, updating, settings, setup and error. The setup screen belongs to the ESP32 phone-setup flow; the simulator does not start a Wi-Fi access point.

Using it from Python

from pathlib import Path

from hermes_gadget.sim import Simulator

sim = Simulator(url="ws://127.0.0.1:8765/gadget", state_dir=Path("/tmp/dev1"))
sim.start()
sim.wait_screen("ready")
sim.type_text("what's the weather?")
sim.wait_for(lambda: sim.last_received("turn.end") is not None, timeout=60)
print(sim.last_received("reply")["text"])
sim.screenshot("reply.png")
sim.close()

tests/test_sim_hub.py and tests/test_gateway_e2e.py are worked examples.