Compare commits

...

20 Commits

Author SHA1 Message Date
alt-glitch ec1714e71f fix(install.ps1): handle uv stderr output with ErrorActionPreference=Stop
On fresh Windows installs, `uv python install` writes download progress to
stderr. With $ErrorActionPreference = 'Stop' (set globally in the script),
PowerShell wraps those stderr lines as ErrorRecord objects when captured via
2>&1, then throws a terminating exception — landing in the catch block even
though uv exits 0 and Python was installed successfully.

Fix: temporarily set ErrorActionPreference to 'Continue' around the native
uv call, then verify success with `uv python find` which is the reliable
signal regardless of exit code / stderr noise.

Tested on Windows 11 (build 26200) with ExecutionPolicy=Restricted,
uv 0.11.11, fresh machine with no prior Python install.
2026-05-08 14:13:06 +05:30
Teknium e0c03defd5 lint: enable PLW1514 as a blocking ruff rule
Turns the existing 'all lints disabled' stance into 'exactly one lint
enabled' — PLW1514 (unspecified-encoding) catches bare open() /
read_text() / write_text() calls that default to locale encoding on
Windows (cp1252), silently corrupting non-ASCII content.

Changes:

1. pyproject.toml
   - Migrate [tool.ruff] top-level select → [tool.ruff.lint].select
     (deprecated config location, ruff was warning on every run)
   - Add preview = true (PLW1514 is a preview rule in ruff 0.15.x)
   - select = ['PLW1514'] (exactly one rule, deliberately minimal)
   - per-file-ignores exempt tests/, plugins/, skills/, optional-skills/ —
     those have their own conventions or intentionally exercise edge cases

2. website/scripts/extract-skills.py
   - Fix 3 remaining bare opens (website/ was excluded from the main
     sweep but needed for ruff check . to go green)

3. tests/test_lint_config.py (new, 5 tests)
   - Guards against accidental rule removal.  If someone deletes PLW1514
     from the select list or disables preview mode, these tests fail
     with a loud message explaining why the rule exists.

Paired with a companion commit (held locally for now, pending a token
with workflow scope) that adds a blocking ruff step to .github/workflows/
lint.yml.  Without that companion commit, ruff is configured correctly
but nothing in CI enforces it yet — the advisory PR comment will still
surface new PLW1514 violations though, so authors see them.

Verified: ruff check . → exit 0, 0 violations across the repo.
Test suite: 90 passed, 14 skipped, 0 failed.
2026-05-07 19:36:13 -07:00
Teknium 9c914c01c8 codebase: add encoding='utf-8' to all bare open() calls (PLW1514)
Closes the last Python-on-Windows UTF-8 exposure by making every
text-mode open() call explicit about its encoding.

Before: on Windows, bare open(path, 'r') defaults to the system
locale encoding (cp1252 on US-locale installs).  That means reading
any config/yaml/markdown/json file with non-ASCII content either
crashes with UnicodeDecodeError or silently mis-decodes bytes.

After: all 89 affected call sites in production code now pass
encoding='utf-8' explicitly.  Works identically on every platform
and every locale, no surprise behavior.

Mechanical sweep via:
  ruff check --preview --extend-select PLW1514 --unsafe-fixes --fix     --exclude 'tests,venv,.venv,node_modules,website,optional-skills,               skills,tinker-atropos,plugins' .

All 89 fixes have the same shape: open(x) or open(x, mode) became
open(x, encoding='utf-8') or open(x, mode, encoding='utf-8').  Nothing
else changed.  Every modified file still parses and the Windows/sandbox
test suite is still green (85 passed, 14 skipped, 0 failed across
tests/tools/test_code_execution_windows_env.py +
tests/tools/test_code_execution_modes.py + tests/tools/test_env_passthrough.py +
tests/test_hermes_bootstrap.py).

Scope notes:
  - tests/ excluded: test fixtures can use locale encoding intentionally
    (exercising edge cases).  If we want to tighten tests later that's
    a separate PR.
  - plugins/ excluded: plugin-specific conventions may differ; plugin
    authors own their code.
  - optional-skills/ and skills/ excluded: skill scripts are user-authored
    and we don't want to mass-edit them.
  - website/ and tinker-atropos/ excluded: vendored / generated content.

46 files touched, 89 +/- lines (symmetric replacement).  No behavior
change on POSIX or on Windows when the file is ASCII; bug fix on
Windows when the file contains non-ASCII.
2026-05-07 19:24:45 -07:00
Teknium 6098272454 hermes_bootstrap: Windows-only UTF-8 stdio shim for all entry points
Codebase-wide fix for Python-on-Windows UTF-8 footguns, complementing
the earlier execute_code sandbox fixes (which remain load-bearing for
when the sandbox explicitly scrubs child env).

Problem: Python on Windows has two long-standing text-encoding pitfalls:

  1. sys.stdout/stderr are bound to the console code page (cp1252 on
     US-locale installs) — print('café') crashes with UnicodeEncodeError.
  2. Subprocess children don't know to use UTF-8 unless PYTHONUTF8 and/or
     PYTHONIOENCODING are set in their env — so any Python we spawn
     (linters, sandbox children, delegation workers) hits the same bug.

Solution: A tiny bootstrap module (hermes_bootstrap.py) imported as the
first statement of every Hermes entry point:

  - hermes_cli/main.py   (hermes / hermes-agent console_script)
  - run_agent.py         (hermes-agent direct)
  - acp_adapter/entry.py (hermes-acp)
  - gateway/run.py       (messaging gateway)
  - batch_runner.py      (parallel batch mode)
  - cli.py               (legacy direct-launch CLI)

On Windows, the bootstrap:
  - os.environ.setdefault('PYTHONUTF8', '1')       (PEP 540 UTF-8 mode)
  - os.environ.setdefault('PYTHONIOENCODING', 'utf-8')
  - sys.stdout/stderr/stdin.reconfigure(encoding='utf-8', errors='replace')

Children inherit the env vars → they run in UTF-8 mode.
Current process's stdio is reconfigured → print('café') works now.

On POSIX (Linux/macOS), the bootstrap is a complete no-op.  We don't
touch LANG, LC_*, or anything else — users who have intentionally
configured a non-UTF-8 locale aren't affected.  POSIX systems are
already UTF-8 by default in 99% of modern setups, so there's nothing
to fix.

setdefault() (not overwrite) means users who explicitly set PYTHONUTF8=0
or PYTHONIOENCODING=cp1252 in their environment are respected.

What this does NOT fix: bare open(path, 'w') calls in the *parent*
process still default to locale encoding because PYTHONUTF8 is only
read at interpreter init.  A ruff PLW1514 sweep (separate follow-up)
will add explicit encoding='utf-8' at those ~219 call sites for
belt-and-suspenders.

Tests (17): 16 passed, 1 skipped on Windows.
  - Windows: env vars set, stdio reconfigured, child inherits UTF-8 mode
  - POSIX: complete no-op (verified on fake POSIX + skipped on real
    POSIX since we don't have a Linux box in this session)
  - Idempotence: multiple calls safe
  - Graceful degradation: non-reconfigurable streams don't crash
  - User opt-out: explicit PYTHONUTF8=0 is respected
  - Load order: every entry point's FIRST top-level import is
    hermes_bootstrap, enforced by an AST-level parametrized test

pyproject.toml: added hermes_bootstrap to py-modules so it ships with
pip installs.
2026-05-07 19:09:40 -07:00
Teknium bf43f6cfdd execute_code: set PYTHONIOENCODING=utf-8 + PYTHONUTF8=1 in child env
Third Windows-specific sandbox bug (after WinError 10106 and the UTF-8
file-write bug): user scripts that print non-ASCII to stdout crash with

    UnicodeEncodeError: 'charmap' codec can't encode character '\u2192'
                        in position N: character maps to <undefined>

Root cause: Python's sys.stdout on Windows is bound to the console code
page (cp1252 on US-locale installs) when the process is attached to a
pipe without PYTHONIOENCODING set.  LLM-generated scripts routinely
print em-dashes, arrows, accented chars, and emoji — all of which cp1252
can't encode.

Fix: spawn the sandbox child with:

    PYTHONIOENCODING=utf-8   # sys.stdin/stdout/stderr all UTF-8
    PYTHONUTF8=1             # PEP 540 UTF-8 mode — open() defaults to UTF-8 too

PYTHONUTF8 is the belt-and-suspenders half: LLM scripts that call
open(path, 'w') without encoding= in user code will now produce UTF-8
files by default, matching what the sandbox already does for its own
staging files.

The parent side already decodes child stdout/stderr as UTF-8 with
errors='replace' (lines 1345-1347) so the end-to-end chain is clean.

On POSIX these values usually match the locale default already, so
setting them is harmless belt-and-suspenders for C/POSIX-locale
containers and minimal base images.

Tests added (4) — total file now at 28 passed, 1 skipped on Windows:
  - test_popen_env_sets_pythonioencoding_utf8 (source grep)
  - test_popen_env_sets_pythonutf8_mode (source grep)
  - test_live_child_can_print_non_ascii (cross-platform live test)
  - test_windows_child_without_utf8_env_would_fail (Windows negative
    control — actually reproduces the bug without our env overrides,
    proving the fix is load-bearing on this system)
2026-05-07 18:59:35 -07:00
Teknium f5ec30dfe6 tests: skip POSIX-venv-layout tests on Windows
test_code_execution_modes.py had two test-level failures and two
class-level stale skip reasons on this Windows-native branch:

  - TestResolveChildPython::test_project_with_virtualenv_picks_venv_python
  - TestResolveChildPython::test_project_prefers_virtualenv_over_conda

Both fail on Windows with OSError: [WinError 1314] — they call
pathlib.Path.symlink_to() to build a fake venv, which requires
developer mode or admin on Windows.  They also assume POSIX venv
layout (bin/python) where Windows uses Scripts/python.exe.  Skip
them with a specific, accurate reason.

Also updated two class-level skipif reasons that said
'execute_code is POSIX-only' — no longer true on this branch.
New reason explains it's the test infrastructure (symlinks + POSIX
venv layout) that's the blocker, not execute_code itself.

Results on Windows Python 3.11:
  Before: 41 passed, 10 skipped, 2 failed
  After:  43 passed, 12 skipped, 0 failed
2026-05-07 18:56:33 -07:00
Teknium 8798bea31f execute_code: write sandbox files as UTF-8 on Windows
Second Windows-specific sandbox bug (WinError 10106 was the first):
after the env-scrub fix let the child start, it immediately failed to
import hermes_tools with:

    SyntaxError: (unicode error) 'utf-8' codec can't decode byte 0x97
                 in position 154: invalid start byte

Root cause: _execute_local wrote the generated hermes_tools.py stub and
the user's script.py via open(path, 'w') without encoding=.  On Windows
the default text-mode encoding is cp1252 (system locale), which encodes
em-dashes (used in the stub's docstrings) as 0x97.  Python then decodes
source files as UTF-8 (PEP 3120) on import, chokes on 0x97, and the
sandbox dies before any tool call.

Fix: pass encoding='utf-8' to all four file opens in the code_execution
path — the two staging writes in _execute_local (hermes_tools.py +
script.py) and the two RPC file-transport reads/writes in the generated
remote stub.  JSON is ASCII-safe for most payloads but tool results
(terminal output, web_extract content) routinely carry non-ASCII.

Tests added (4):
  - test_stub_and_script_writes_specify_utf8 — source grep guard
  - test_file_rpc_stub_uses_utf8 — generated remote stub check
  - test_stub_source_roundtrips_through_utf8 — concrete round-trip
  - test_windows_default_encoding_would_have_failed — negative control
    (skips on modern Python builds where default is already UTF-8
    compatible, but retained for platforms where the regression could
    return)

24/25 tests pass on Windows 3.11 (negative control skips because this
Python build handles em-dashes via cp1252 subset — the fix is still
correct, just the corruption path isn't always triggerable).
2026-05-07 18:52:59 -07:00
Teknium 668e4b8d7e tests: lock in POSIX-equivalence guard for execute_code env scrubber
Adds TestPosixEquivalence to test_code_execution_windows_env.py.  The
class pins the invariant that _scrub_child_env(env, is_windows=False)
produces byte-for-byte identical output to the pre-refactor inline
scrubber, across a matrix of:

  - 2 synthetic envs (POSIX-shaped, Windows-shaped-on-POSIX)
  - 3 passthrough rules (none, single-var, everything)
  - 1 real-os.environ check on whatever platform runs the test

Plus a superset sanity check: is_windows=True must keep everything
is_windows=False keeps, and any extras must come from the
_WINDOWS_ESSENTIAL_ENV_VARS allowlist.

Rationale: the previous commit refactored the env-scrubbing inline
block into a helper.  Future changes to that helper must not silently
regress POSIX behavior — if someone needs to change it, they update
_legacy_posix_scrubber in lockstep so the churn is visible in review.

All 21 tests in the file pass locally on Windows (pytest 9.0.3).  8 of
them are parametrized equivalence checks that run on every OS.
2026-05-07 18:45:34 -07:00
Teknium fab984c7f8 execute_code: pass through Windows OS-essential env vars
The sandbox's env scrubbing was dropping SYSTEMROOT, WINDIR, COMSPEC,
APPDATA, etc. On Windows this broke the child process before any RPC
could happen:

    OSError: [WinError 10106] The requested service provider could not
    be loaded or initialized

Python's socket module uses SYSTEMROOT to locate mswsock.dll during
Winsock initialization. Without it, socket.socket(AF_INET, SOCK_STREAM)
fails — and the existing loopback-TCP fallback for Windows couldn't work.

Fix: add a small Windows-only allowlist (_WINDOWS_ESSENTIAL_ENV_VARS)
matched by exact uppercase name, after the existing secret-substring
block. The secret block still runs first, so the allowlist cannot be
used to exfiltrate credentials. Also extract the env scrubber into a
testable helper (_scrub_child_env) that takes is_windows as a parameter,
so the logic can be unit-tested on any OS.

Live Winsock smoke test verifies that a child spawned with the scrubbed
env can now create an AF_INET socket on a real Windows host; the test
is guarded by sys.platform == 'win32' so POSIX CI stays green.
2026-05-07 18:39:38 -07:00
Teknium f0d2516a30 fix(windows): prefer npm.cmd over npm.ps1, skip .py argv0 in relaunch
Two fixes from teknium1's next install run:

1. **npm install: "npm.ps1 cannot be loaded because running scripts is
   disabled on this system."**  Get-Command's default PATHEXT ordering
   picked up ``npm.ps1`` (the PowerShell shim) ahead of ``npm.cmd`` (the
   batch shim).  Most Windows users have PowerShell's execution policy
   set to Restricted or RemoteSigned, which blocks unsigned ``.ps1``
   files.  ``npm.cmd`` has no such restriction and works universally.

   Install-NodeDeps now detects when Get-Command returned npm.ps1, looks
   for a sibling npm.cmd in the same directory, and prefers it.  Prints
   an info line so the user sees why.  Emits a warning + hint if only
   npm.ps1 is available.

2. **"Launch hermes chat now? Y" crashes with "%1 is not a valid Win32
   application" on Windows installs.**  The setup wizard calls
   ``relaunch(["chat"])``; ``resolve_hermes_bin()`` returned
   ``sys.argv[0]`` which was ``...\\hermes_cli\\main.py`` (because hermes
   was launched via ``python -m hermes_cli.main`` during setup).

   On Windows, ``os.access(script.py, os.X_OK)`` returns True because
   PATHEXT lists ``.py`` when the Python launcher is registered — but
   ``subprocess.run([script.py, ...])`` can't actually execute a ``.py``
   directly.  CreateProcessW needs a real PE file.

   Fixed ``resolve_hermes_bin`` to reject ``.py``/``.pyc`` argv0 values
   on Windows specifically.  Falls through to ``shutil.which("hermes")``
   (hermes.exe in the venv Scripts dir) or, as a final fallback, lets
   build_relaunch_argv build ``[sys.executable, "-m", "hermes_cli.main"]``
   which is bulletproof.  POSIX behaviour unchanged — ``.py`` argv0 with
   a shebang + chmod+x is still a valid exec target there.

3 new tests cover the Windows paths: .py argv0 + hermes.exe on PATH →
returns hermes.exe; .py argv0 + no PATH → returns None (caller uses
python -m); POSIX + executable .py → still accepted.

26 relaunch tests pass, no POSIX regressions.
2026-05-07 18:29:17 -07:00
Teknium 2e403bd0a4 fix(windows): enable execute_code — stale AF_UNIX gate was blocking the tool
teknium1 noticed execute_code was missing from his enabled tools on Windows.
Root cause: tools/code_execution_tool.py set ``SANDBOX_AVAILABLE =
sys.platform != \"win32\"`` as a module-level constant, originally because
the RPC transport required AF_UNIX.  We added loopback TCP fallback for
the sandbox in commit eeb723fff (and covered it in the Windows TCP tests),
but forgot to lift the availability gate.  So execute_code was still
invisible via the check_fn path on Windows.

- SANDBOX_AVAILABLE is now True unconditionally (it's still checked — a
  future platform could flip it off via monkeypatch/env if needed).
- Error message when disabled no longer mentions Windows specifically,
  just says 'sandbox is unavailable in this environment'.
- test_windows_returns_error updated: patches SANDBOX_AVAILABLE=False
  directly (which was always its real intent) and asserts on 'unavailable'
  instead of 'Windows'.

Tests: 171 code-execution + windows-compat tests pass, no regressions.
2026-05-07 18:17:31 -07:00
Teknium 2c7b479d16 fix(windows): %1 install error, patch CRLF false-negative, SOUL.md BOM
Three bugs from teknium1's successful install + diagnostic chat on Windows:

1. **Start-Process -FilePath npm.cmd fails with "%1 is not a valid Win32
   application".**  Start-Process bypasses cmd.exe and PATHEXT to call
   CreateProcessW directly, which refuses .cmd batch shims.  Switched
   Install-NodeDeps to use PowerShell's invocation operator (``& $npmExe
   install --silent *> $log``) which DOES honour PATHEXT.  Extracted a
   ``_Run-NpmInstall`` helper so the browser + TUI paths share the same
   logic.  Captures $LASTEXITCODE correctly, still surfaces the real
   stderr on failure with a log-file pointer for the full output.

2. **patch tool returns false-negative on Windows due to CRLF round-trip.**
   Root cause was upstream of patch: ``subprocess.Popen(..., text=True,
   stdin=PIPE)`` on Windows translates ``\\n`` → ``\\r\\n`` when data flows
   through the stdin pipe.  ``_pipe_stdin()`` was writing the patch's
   new_content string through a text-mode pipe, bash then wrote those
   CRLF bytes to disk, and patch's post-write verify compared the
   on-disk CRLF bytes against the original LF-only string — fail.

   Fixed in two places for defense in depth:
   - ``_pipe_stdin()`` now writes through ``proc.stdin.buffer`` with
     explicit UTF-8 encoding, bypassing Python's newline translation on
     every platform.  No behaviour change on POSIX (bytes are identical)
     but stops the CRLF injection on Windows.
   - ``patch_replace``'s post-write verify normalizes CRLF→LF on both
     sides before comparing, so even if some future backend still
     translates newlines the patch tool won't report a bogus failure.

3. **SOUL.md gets a UTF-8 BOM on Windows PowerShell 5.1.**  ``Set-Content
   -Encoding UTF8`` on PS5.1 writes UTF-8 WITH a byte-order-mark (changed
   in PS7 via ``utf8NoBOM``).  Hermes's prompt-injection scanner sees
   the BOM (U+FEFF invisible char) and refuses to load the file, so
   SOUL.md's persona instructions never get applied.

   Fixed by writing the file via ``[System.IO.File]::WriteAllText``
   with an explicit ``UTF8Encoding($false)`` — BOM-free on every
   PowerShell version.

All POSIX behaviour verified unchanged: 198 tests pass across
test_file_operations, test_local_env_cwd_recovery, test_code_execution,
test_windows_native_support, test_windows_compat.
2026-05-07 18:11:43 -07:00
Teknium 225b57f314 fix(install.ps1): step out of $InstallDir before touching it + harden repo probe
User hit 'fatal: not in a git directory' on re-install because:

1. They ran Remove-Item -Force $env:LOCALAPPDATA\hermes -ErrorAction
   SilentlyContinue WHILE cd'd inside the install dir.  Windows
   silently refuses to delete a directory any shell is currently cd'd
   inside and leaves the skeleton intact, but the -ErrorAction
   SilentlyContinue swallowed every partial-delete failure so they
   thought the wipe succeeded.

2. The installer then walked into Install-Repository, saw $InstallDir
   still exists with a partial .git stub, my repo-validity probe
   returned success (the probe's git rev-parse may have exit-code-zeroed
   in a way I didn't expect), and the real git fetch died with three
   'fatal: not a git repository' errors.

Two fixes belt-and-braces:

- Main() now cds to $env:USERPROFILE at start if the current shell
  is inside $InstallDir.  Harmless when the user ran from elsewhere;
  critical when they didn't.  This alone fixes the user's case.

- Install-Repository's 'is this a valid repo' probe now runs BOTH
  git rev-parse --is-inside-work-tree AND git status, resets
  $LASTEXITCODE before each to avoid picking up a stale 0, and
  requires BOTH to succeed.  Also requires rev-parse's output to
  match 'true' (not just exit 0) to rule out exit-0-with-empty-output
  edge cases.
2026-05-07 18:05:35 -07:00
Teknium 4d7e72e14d fix(install.ps1): validate existing repo via git itself + clean up broken stubs
teknium1 hit "fatal: not in a git directory" on re-install when the previous
install left a $InstallDir\.git stub that Test-Path matched but git didn't
recognize (three "fatal: not a git repository" lines, then the script
exited before touching anything).

Two bugs:

1. Test-Path "$InstallDir\.git" was a weak gate — it matches .git
   whether it's a directory, file, symlink, submodule gitfile, OR a
   broken stub from a failed previous Remove-Item.  Replaced with a
   real repo probe: Push-Location + git rev-parse --is-inside-work-tree
   + $LASTEXITCODE check.  If git itself can't see a repo, we treat
   the directory as not-a-repo and fall through to fresh clone.

2. The original update path ignored $LASTEXITCODE.  fetch/checkout/pull
   all emitted fatals but the script kept going.  Now each command
   checks $LASTEXITCODE and throws with an explicit message.

Also: when the directory exists but isn't a valid repo, the new code
wipes it (Remove-Item -ErrorAction Stop) and falls through to fresh
clone, instead of dying with the old "Directory exists but is not a git
repository" error.  If the wipe itself fails (file locked, hermes still
running), we throw with a user-readable "close any programs using files
in <dir>" hint.

Refactored the function to use a $didUpdate flag instead of my earlier
draft's early `return` — that was skipping the submodule init block at
the bottom of the function.  Both the update and fresh-clone paths now
fall through to the submodule init step, which is correct (git pull
doesn't auto-update submodules).

PowerShell structural check: 21 functions defined, braces balanced.
2026-05-07 18:00:59 -07:00
Teknium 787d964ea1 fix(windows): quote cache paths in bash + augment PATH so rg/bash resolve on first launch
Three interrelated bugs from teknium1's first interactive chat on Windows:

1. **Snapshot/cwd file paths unquoted in bash command strings.**  The session
   bootstrap and per-command wrapper interpolated
   ``self._snapshot_path`` / ``self._cwd_file`` unquoted into bash commands
   like ``export -p > C:/Users/ryanc/.../hermes-snap-xxx.sh``.  Git Bash's
   MSYS2 layer handles ``C:/...`` paths correctly ONLY when quoted; unquoted,
   the colon and forward-slash get glob-parsed and the redirect targets a
   bogus path.  Symptom: every terminal command emitted two
   ``C:/Users/.../hermes-snap-*.sh (No such file or directory)`` lines that
   bled into stdout (``stderr=STDOUT`` on the local backend) and corrupted
   file contents when the agent wrote to scratch paths via the terminal
   tool.  Fix: ``shlex.quote()`` every interpolation of ``_snapshot_path``
   and ``_cwd_file`` in base.py — no-op on POSIX (the paths contain no
   shell-metachars), critical on Windows.

2. **Stale PATH on first hermes launch after install.**  ``install.ps1``
   adds the PortableGit ``cmd`` / ``bin`` / ``usr\bin`` directories to the
   Windows **User** PATH via ``SetEnvironmentVariable(..., "User")``.  That
   write propagates to newly *spawned* processes only — already-running
   shells (including the one the user types ``hermes`` into immediately
   after install) retain their old PATH.  So hermes starts with a PATH that
   doesn't include bash, rg, grep, ssh — and ``search_files`` reports
   "rg/find not available" when the user clearly just installed them.

   Fix: new ``_augment_path_with_known_tools()`` helper called from
   ``configure_windows_stdio()`` on startup.  Prepends the Hermes-managed
   Git directories + the WinGet Links directory (where ripgrep lands) to
   ``os.environ['PATH']`` if they exist on disk but aren't already in
   PATH.  Subsequent subprocess calls (including bash spawns via
   ``_find_bash()``) inherit the augmented PATH and find everything.
   No-op on POSIX and when the directories don't exist.

3. **Root cause of "file content corruption".**  #1 was the proximate cause.
   Errors like ``C:/Users/.../hermes-snap-xxx.sh: No such file or directory``
   were emitted on stderr by the failed redirect, captured into stdout via
   ``stderr=subprocess.STDOUT``, and if the agent used terminal commands
   like ``cat > file`` the leaked error bytes became part of the file.
   Fixing #1 eliminates this entirely.

## Tests

All 77 Windows-compat tests still pass on Linux (POSIX path is
shlex.quote('/tmp/foo.sh') → '/tmp/foo.sh' — unchanged).

## Not addressed here (would need a bigger design)

- Python file tools (``write_file``, ``read_file``) and the bash-backed
  terminal tool see DIFFERENT views of ``/tmp`` on Windows.  Python treats
  ``/tmp`` as ``C:\tmp`` (drive-relative), Git Bash's MSYS2 treats it as
  a virtual mount to the PortableGit install's ``tmp\``.  Would need a
  translation shim in the Python tools to resolve bash-virtual paths to
  their native-Windows equivalents.  Workaround for users today: use
  absolute native paths (``C:\Users\you\...``) instead of ``/tmp/...``
  when crossing between terminal and Python file tools.
2026-05-07 17:51:57 -07:00
Teknium cf9b2df57a fix(windows): use PortableGit (not MinGit), fix relaunch os.execvp crash, surface npm errors
Three real bugs from teknium1's first Windows install run:

1. **MinGit has no bash.exe.**  MinGit is the minimal-automation Git for Windows
   distribution — it ships git.exe but deliberately strips bash and the POSIX
   coreutils.  Installer logged "Could not locate bash.exe" and Hermes would
   fail to run any shell command.  Switched to PortableGit — the full Git for
   Windows minus the installer UI.  PortableGit ships bash.exe at
   <root>\bin\bash.exe plus sh, awk, sed, grep, curl, ssh in usr\bin\.  ARM64
   variant is detected separately (PortableGit-*-arm64.7z.exe).  32-bit falls
   back to MinGit-32-bit with a warning (PortableGit is 64-bit only).

   PortableGit ships as a 7z self-extractor (56MB vs MinGit's 38MB).  We
   invoke it with `-o<target> -y` to extract silently — no 7z install needed,
   it's self-contained.

   Updated tools/environments/local.py::_find_bash candidate order to prefer
   the PortableGit layout (<root>\bin\bash.exe) with the MinGit layout
   (<root>\usr\bin\bash.exe) as a fallback so existing installs keep working.

2. **os.execvp "Exec format error" on Windows.**  Setup wizard's "Launch
   hermes chat now? Y" called `os.execvp(["hermes", "chat"])` which on
   Windows can only swap to real Win32 .exe files — chokes with OSError(8)
   on .cmd batch shims and Python console-script wrappers.  Added a
   win32 branch in hermes_cli/relaunch.py::relaunch() that uses
   subprocess.run + sys.exit — functionally identical (user sees "hermes
   exited, then new hermes started") with one extra PID in play.  POSIX
   path is UNCHANGED — still uses os.execvp for in-place replacement.
   Catches OSError in the Windows branch and surfaces a "open a new
   terminal so PATH picks up, then re-run hermes" hint instead of a
   cryptic traceback.

3. **npm install failures silent on Windows.**  The install.ps1 was invoking
   `npm install --silent 2>&1 | Out-Null` inside a try/catch.  PowerShell's
   try/catch does NOT trigger on non-zero process exit codes — only on
   unhandled .NET exceptions — so npm failing printed a generic "npm
   install failed" with zero information about WHY.  The silent pipe ate
   the stderr.

   Rewrote Install-NodeDeps to:
   - Resolve npm.cmd via Get-Command (respects PATHEXT) instead of
     relying on bare `npm` name resolution.
   - Use Start-Process with -PassThru to capture the actual exit code.
   - Redirect stderr to a temp log and surface the first ~800 chars of
     the real npm error when install fails, plus the log path for the
     full text.
   - Fail loudly with the right exit code instead of a misleading success.
   - Bail cleanly with a helpful message when npm isn't on PATH at all.

4. **"True" printing to console after Node check.**  `Test-Node` returns $true;
   installer called it as a bare statement (no assignment, no cast).  PowerShell
   prints bare return values.  Wrapped the call in `[void](Test-Node)`.

## Tests

- Added 3 new tests in tests/hermes_cli/test_relaunch.py covering the
  Windows branch: subprocess is called (not execvp), child exit code
  propagates, OSError surfaces a helpful message.  All 23 tests pass
  (20 existing + 3 new).
- 77 Windows-compat tests still pass, POSIX behaviour unchanged.
2026-05-07 17:42:47 -07:00
Teknium eeb723fff2 feat(windows): close remaining POSIX-only landmines — TUI crash, kanban waitpid, AF_UNIX sandbox, /bin/bash, npm .cmd shims, cwd tracking, detach flags
Second pass on native Windows support, driven by a systematic audit across
five areas: POSIX-only primitives (signal.SIGKILL/SIGHUP/SIGPIPE, os.WNOHANG,
os.setsid), path translation bugs (/c/Users → C:\Users), subprocess patterns
(npm.cmd batch shims, start_new_session no-op on Windows), subsystem health
(cron, gateway daemon, update flow), and module-level import guards.

Every change is platform-gated — POSIX (Linux/macOS) behaviour is preserved
bit-identical. Explicit "do no harm" test: test_posix_path_preserved_on_linux,
test_posix_noop, test_windows_detach_popen_kwargs_is_posix_equivalent_on_posix.

## New module

- hermes_cli/_subprocess_compat.py — shared helpers (resolve_node_command,
  windows_detach_flags, windows_hide_flags, windows_detach_popen_kwargs).
  All no-ops on non-Windows.

## CRITICAL fixes (would crash or silently break on Windows)

- tui_gateway/entry.py: SIGPIPE/SIGHUP referenced at module top level would
  AttributeError on import on Windows, breaking `hermes --tui` entirely (it
  spawns this module as a subprocess).  Guard each signal.signal() call with
  hasattr() and add SIGBREAK as Windows' SIGHUP equivalent.

- hermes_cli/kanban_db.py: os.waitpid(-1, os.WNOHANG) in dispatcher tick was
  unguarded.  os.WNOHANG doesn't exist on Windows.  Gate the whole reap loop
  behind `os.name != "nt"` — Windows has no zombies anyway.

- tools/code_execution_tool.py: AF_UNIX socket for execute_code RPC fails on
  most Windows builds.  Fall back to loopback TCP (AF_INET on 127.0.0.1:0
  ephemeral port) when _IS_WINDOWS.  HERMES_RPC_SOCKET env var now accepts
  either a filesystem path (POSIX) or `tcp://127.0.0.1:<port>` (Windows).
  Generated sandbox client parses both.

- cron/scheduler.py: `argv = ["/bin/bash", str(path)]` hardcoded.  Use
  shutil.which("bash") so Windows (Git Bash via MinGit) works, with a
  readable error when bash is genuinely absent.

- 6 bare npm/npx spawn sites: tools_config.py x2, doctor.py, whatsapp.py
  (npm install + node version probe), browser_tool.py x2.  On Windows npm
  is npm.cmd / npx is npx.cmd (batch shims); subprocess.Popen(["npm", ...])
  fails with WinError 193.  shutil.which(...) returns the absolute .cmd
  path which CreateProcessW accepts because the extension routes through
  cmd.exe /c.  POSIX behaviour unchanged (shutil.which still returns the
  same path subprocess would resolve itself).

## HIGH fixes (silent misbehaviour on Windows)

- tools/environments/local.py get_temp_dir: hardcoded /tmp returned on
  Windows meant `_cwd_file = "/tmp/hermes-cwd-*.txt"`, which bash wrote
  via MSYS2's virtual /tmp but native Python couldn't open.  Result: cwd
  tracking silently broken — `cd` in terminal tool did nothing.  Windows
  branch now returns `%HERMES_HOME%/cache/terminal` with forward slashes
  (works in both bash and Python, guaranteed no spaces).

- tools/environments/local.py _make_run_env PATH injection: `/usr/bin not
  in split(":")` heuristic mangles Windows PATH (";" separator).  Gate
  the injection behind `not _IS_WINDOWS`.

- hermes_cli/gateway.py launch_detached_profile_gateway_restart: outer
  Popen + watcher-script Popen both used start_new_session=True, which
  Windows silently ignores.  Watcher stayed attached to CLI's console,
  died when user closed terminal after `hermes update`, left gateway
  stale.  Now branches through windows_detach_popen_kwargs() helper
  (CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS | CREATE_NO_WINDOW on
  Windows, start_new_session=True on POSIX — identical to main).

## MEDIUM fixes

- gateway/run.py /restart and /update handlers: hardcoded bash/setsid
  chain crashes on Windows when user triggers /update in-gateway.  Now
  has sys.platform=="win32" branch using sys.executable + a tiny
  Python watcher with proper detach flags.  POSIX path is unchanged.

- cli.py _git_repo_root: Git on Windows sometimes returns /c/Users/...
  style paths that break subprocess.Popen(cwd=...) and Path().resolve().
  Added _normalize_git_bash_path() helper that translates /c/Users,
  /cygdrive/c, /mnt/c variants to native C:\Users form.  POSIX no-op.
  _git_repo_root() now routes every result through it.

- cli.py worktree .worktreeinclude: os.symlink on directories failed
  hard on Windows (requires admin or Developer Mode).  Falls back to
  shutil.copytree with a warning log.

## Tests

- 29 new tests in tests/tools/test_windows_native_support.py covering:
  subprocess_compat helpers, TUI entry signal guards, kanban waitpid
  guard, code_execution TCP fallback source-level invariants, cron bash
  resolution, npm/npx bare-spawn lint per-file, local env Windows temp
  dir, PATH injection gating, git bash path normalization, symlink
  fallback, gateway detached watcher flags.

- One existing test assertion adjusted in test_browser_homebrew_paths:
  it compared captured Popen argv to the BARE `"npx"` literal; after the
  shutil.which() change argv[0] is the absolute path.  New assertion
  checks the shape (two items, second is `agent-browser`) rather than
  the exact first-item string.  Behaviour unchanged; test was too strict.

All 56 tests pass on Linux (30 from previous commits + 26 new).
267 tests from the affected files/dirs (browser, code_exec, local_env,
process_registry, kanban_db, windows_compat) all pass — zero regressions.
tests/hermes_cli/ (3909 pass) and tests/gateway/ (5021 pass) unchanged;
all pre-existing test failures confirmed unrelated via `git stash` re-run.

## What's still deferred (LOW priority)

- Visible cmd-window flashes on short-lived console apps (~14 sites) —
  cosmetic, needs a follow-up pass once we have user reports.
- agent/file_safety.py POSIX-only security deny patterns — separate
  hardening task.
- tools/process_registry.py returning "/tmp" as fallback — theoretical;
  reachable only when all env-var candidates fail.
2026-05-07 17:29:31 -07:00
Teknium 1da89528e7 fix(windows-editor): default EDITOR=notepad so /edit and Ctrl+X Ctrl+E work
Pre-existing Windows bug surfaced while reviewing the portable-MinGit
install: prompt_toolkit's Buffer.open_in_editor() falls back to POSIX
absolute paths (/usr/bin/nano, /usr/bin/vi, /usr/bin/emacs) that don't
exist on native Windows.  When neither $EDITOR nor $VISUAL is set,
Ctrl+X Ctrl+E ("open prompt in editor") and /edit both silently do
nothing on Windows — the user hits the key, nothing happens, no error.

This wasn't caused by MinGit (full Git for Windows doesn't fix it either,
because the Windows Python subprocess call resolves `/usr/bin/nano` as
`C:\usr\bin\nano`, which doesn't exist even with nano installed).

Fixes:
- hermes_cli/stdio.py::configure_windows_stdio now sets EDITOR=notepad
  on Windows if neither EDITOR nor VISUAL is set.  notepad.exe is in
  every Windows install, works as a blocking editor (subprocess.call
  waits for the window to close), and writes back to the file.
- hermes_cli/config.py (hermes config edit): reorder fallback list so
  Windows tries notepad first — previously nano led the list, which
  required Git Bash / WSL to be in PATH.
- Users who want VSCode / Neovim / Notepad++ can still override via
  $env:EDITOR — that's checked before our default kicks in.  Docstring
  spells out the common overrides.

The Ink TUI (`hermes --tui`) already handled Windows correctly via
ui-tui/src/lib/editor.ts falling back to notepad.exe on win32 — this
commit brings the classic prompt_toolkit CLI into parity.

3 new tests in test_windows_native_support.py verify:
- EDITOR=notepad gets set when unset on Windows
- Explicit $EDITOR is respected
- $VISUAL is respected (not overwritten by our default)
2026-05-07 16:46:37 -07:00
Teknium 5486ad2f2a feat(windows-install): bundle portable MinGit instead of relying on winget
User hit a real failure case: their system Git was in a half-installed state
(can neither uninstall nor reinstall) and winget refused to work around it.
We were one step away from shipping an installer that would have left users
with exactly the problem he already had.

What other agents do (reality check):
- Claude Code: requires pre-installed Git; breaks if user doesn't have it.
- OpenCode, Codex: don't need bash at all — PowerShell-first design.
- Cline: uses whatever shell VSCode is configured with; installs nothing.

None of them solve the "broken system Git" problem.  We need to own our Git.

Changes:
- scripts/install.ps1::Install-Git: dropped winget path entirely.  Now:
  (1) use existing git if present; (2) download portable MinGit from the
  official git-for-windows GitHub release to %LOCALAPPDATA%\hermes\git.
  No winget, no admin, no Windows installer registry, no system impact.
- Added %LOCALAPPDATA%\hermes\git\{cmd,usr\bin} to User PATH so git + bash
  + POSIX coreutils (which, env, grep, …) resolve in fresh shells.
- tools/environments/local.py::_find_bash: reorder so Hermes' portable
  MinGit install is checked BEFORE falling through to shutil.which("bash")
  or system install locations.  This way a broken system Git can't
  hijack the bash lookup.
- README + installation docs reworded to reflect the new story: "portable
  Git Bash, isolated from any system install, recoverable via rm -rf if it
  ever breaks."

Recoverability: if Hermes' Git install ever breaks, ``Remove-Item %LOCALAPPDATA%\hermes\git``
and re-run the installer — no system impact, no uninstall drama, no winget
to fight with.
2026-05-07 16:38:11 -07:00
Teknium fda234a210 feat(windows): close native-Windows install gaps — crash-free startup, UTF-8 stdio, tzdata dep, docs
Native Windows (with Git for Windows installed) can now run the Hermes CLI
and gateway end-to-end without crashing.  install.ps1 already existed and
the Git Bash terminal backend was already wired up — this PR fills the
remaining gaps discovered by auditing every Windows-unsafe primitive
(`signal.SIGKILL`, `os.kill(pid, 0)` probes, bare `fcntl`/`termios`
imports) and by comparing hermes against how Claude Code, OpenCode, Codex,
and Cline handle native Windows.

## What changed

### UTF-8 stdio (new module)
- `hermes_cli/stdio.py` — single `configure_windows_stdio()` entry point.
  Flips the console code page to CP_UTF8 (65001), reconfigures
  `sys.stdout`/`stderr`/`stdin` to UTF-8, sets `PYTHONIOENCODING` + `PYTHONUTF8`
  for subprocesses.  No-op on non-Windows.  Opt out via `HERMES_DISABLE_WINDOWS_UTF8=1`.
- Called early in `cli.py::main`, `hermes_cli/main.py::main`, and
  `gateway/run.py::main` so Unicode banners (box-drawing, geometric
  symbols, non-Latin chat text) don't `UnicodeEncodeError` on cp1252
  consoles.

### Crash sites fixed
- `hermes_cli/main.py:7970` (hermes update → stuck gateway sweep): raw
  `os.kill(pid, _signal.SIGKILL)` → `gateway.status.terminate_pid(pid, force=True)`
  which routes through `taskkill /T /F` on Windows.
- `hermes_cli/profiles.py::_stop_gateway_process`: same fix — also
  converted SIGTERM path to `terminate_pid()` and widened OSError catch
  on the intermediate `os.kill(pid, 0)` probe.
- `hermes_cli/kanban_db.py:2914, 3041`: raw `signal.SIGKILL` →
  `getattr(signal, "SIGKILL", signal.SIGTERM)` fallback (matches the
  pattern already used in `gateway/status.py`).

### OSError widening on `os.kill(pid, 0)` probes
Windows raises `OSError` (WinError 87) for a gone PID instead of
`ProcessLookupError`.  Widened the catch at:
- `gateway/run.py:15101` (`--replace` wait-for-exit loop — without this,
  the loop busy-spins the full 10s every Windows gateway start)
- `hermes_cli/gateway.py:228, 460, 940`
- `hermes_cli/profiles.py:777`
- `tools/process_registry.py::_is_host_pid_alive`
- `tools/browser_tool.py:1170, 1206`

### Dashboard PTY graceful degradation
`hermes_cli/pty_bridge.py` depends on `fcntl`/`termios`/`ptyprocess`,
none of which exist on native Windows.  Previously a Windows dashboard
would crash on `import hermes_cli.web_server` because of a top-level
import.  Now:
- `hermes_cli/web_server.py` wraps the pty_bridge import in
  `try/except ImportError` and sets `_PTY_BRIDGE_AVAILABLE=False`.
- The `/api/pty` WebSocket handler returns a friendly "use WSL2 for
  this tab" message instead of exploding.
- Every other dashboard feature (sessions, jobs, metrics, config
  editor) runs natively on Windows.

### Dependency
- `pyproject.toml`: add `tzdata>=2023.3; sys_platform == 'win32'` so
  Python's `zoneinfo` works on Windows (which has no IANA tzdata
  shipped with the OS).  Credits @sprmn24 (PR #13182).

### Docs
- README.md: removed "Native Windows is not supported"; added
  PowerShell one-liner and Git-for-Windows prerequisite note.
- `website/docs/getting-started/installation.md`: new Windows section
  with capability matrix (everything native except the dashboard
  `/chat` PTY tab, which is WSL2-only).
- `website/docs/user-guide/windows-wsl-quickstart.md`: reframed as
  "WSL2 as an alternative to native" rather than "the only way".
- `website/docs/developer-guide/contributing.md`: updated
  cross-platform guidance with the `signal.SIGKILL` / `OSError`
  rules we enforce now.
- `website/docs/user-guide/features/web-dashboard.md`: acknowledged
  native Windows works for everything except the embedded PTY pane.

## Why this shape

Pulled from a survey of how other agent codebases handle native
Windows (Claude Code, OpenCode, Codex, Cline):

- All four treat Git Bash as the canonical shell on Windows, same as
  hermes already does in `tools/environments/local.py::_find_bash()`.
- None of them force `SetConsoleOutputCP` — but they don't have to,
  Node/Rust write UTF-16 to the Win32 console API.  Python does not get
  that for free, so we flip CP_UTF8 via ctypes.
- None of them ship PowerShell-as-primary-shell (Claude Code exposes
  PS as a secondary tool; scope creep for this PR).
- All of them use `taskkill /T /F` for force-kill on Windows, which
  is exactly what `gateway.status.terminate_pid(force=True)` does.

## Non-goals (deliberate scope limits)

- No PowerShell-as-a-second-shell tool — worth designing separately.
- No terminal routing rewrite (#12317, #15461, #19800 cluster) — that's
  the hardest design call and needs a separate doc.
- No wholesale `open()` → `open(..., encoding="utf-8")` sweep (Tianworld
  cluster) — will do as follow-up if users hit actual breakage; most
  modern code already specifies it.

## Validation

- 28 new tests in `tests/tools/test_windows_native_support.py` — all
  platform-mocked, pass on Linux CI.  Cover:
  - `configure_windows_stdio` idempotency, opt-out, env-preservation
  - `terminate_pid` taskkill routing, failure → OSError, FileNotFoundError fallback
  - `getattr(signal, "SIGKILL", …)` fallback shape
  - `_is_host_pid_alive` OSError widening (Windows-gone-PID behavior)
  - Source-level checks that all entry points call `configure_windows_stdio`
  - pty_bridge import-guard present in `web_server.py`
  - README no longer says "not supported"
- 12 pre-existing tests in `tests/tools/test_windows_compat.py` still pass.
- `tests/hermes_cli/` ran fully (3909 passed, 9 failures — all confirmed
  pre-existing on main by stash-test).
- `tests/gateway/` ran fully (5021 passed, 1 pre-existing failure).
- `tests/tools/test_process_registry.py` + `test_browser_*` pass.
- Manual smoke: `import hermes_cli.stdio; import gateway.run;
  import hermes_cli.web_server` — all clean, `_PTY_BRIDGE_AVAILABLE=True`
  on Linux (as expected).

## Files

- New: `hermes_cli/stdio.py`, `tests/tools/test_windows_native_support.py`
- Modified: `cli.py`, `gateway/run.py`, `hermes_cli/main.py`,
  `hermes_cli/profiles.py`, `hermes_cli/gateway.py`,
  `hermes_cli/kanban_db.py`, `hermes_cli/pty_bridge.py`,
  `hermes_cli/web_server.py`, `tools/browser_tool.py`,
  `tools/process_registry.py`, `pyproject.toml`, `README.md`, and 4
  docs pages.

Credits to everyone whose prior PR work informed these fixes — see
the co-author trailers.  All of the PRs listed in
`~/.hermes/plans/windows-support-prs.md` fixing `os.kill` / `signal.SIGKILL`
/ UTF-8 stdio / tzdata / README patterns found the same issues; this PR
consolidates them.

Co-authored-by: Philip D'Souza <9472774+PhilipAD@users.noreply.github.com>
Co-authored-by: Arecanon <42595053+ArecaNon@users.noreply.github.com>
Co-authored-by: XiaoXiao0221 <263113677+XiaoXiao0221@users.noreply.github.com>
Co-authored-by: Lars Hagen <1360677+lars-hagen@users.noreply.github.com>
Co-authored-by: Luan Dias <65574834+luandiasrj@users.noreply.github.com>
Co-authored-by: Ruzzgar <ruzzgarcn@gmail.com>
Co-authored-by: sprmn24 <oncuevtv@gmail.com>
Co-authored-by: adybag14-cyber <252811164+adybag14-cyber@users.noreply.github.com>
Co-authored-by: Prasanna28Devadiga <54196612+Prasanna28Devadiga@users.noreply.github.com>
2026-05-07 16:31:40 -07:00
76 changed files with 4086 additions and 320 deletions
+14 -2
View File
@@ -30,15 +30,27 @@ Use any model you want — [Nous Portal](https://portal.nousresearch.com), [Open
## Quick Install
### Linux, macOS, WSL2, Termux
```bash
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
```
Works on Linux, macOS, WSL2, and Android via Termux. The installer handles the platform-specific setup for you.
### Windows (native, PowerShell)
Run this in PowerShell:
```powershell
irm https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.ps1 | iex
```
The installer handles everything: uv, Python 3.11, Node.js, ripgrep, ffmpeg, **and a portable Git Bash** (MinGit, unpacked to `%LOCALAPPDATA%\hermes\git` — no admin required, completely isolated from any system Git install). Hermes uses this bundled Git Bash to run shell commands.
If you already have Git installed, the installer detects it and uses that instead. Otherwise a ~45MB MinGit download is all you need — it won't touch or interfere with any system Git.
> **Android / Termux:** The tested manual path is documented in the [Termux guide](https://hermes-agent.nousresearch.com/docs/getting-started/termux). On Termux, Hermes installs a curated `.[termux]` extra because the full `.[all]` extra currently pulls Android-incompatible voice dependencies.
>
> **Windows:** Native Windows is not supported. Please install [WSL2](https://learn.microsoft.com/en-us/windows/wsl/install) and run the command above.
> **Windows:** Native Windows is supported — the PowerShell one-liner above installs everything. If you'd rather use WSL2, the Linux command works there too. Native Windows install lives under `%LOCALAPPDATA%\hermes`; WSL2 installs under `~/.hermes` as on Linux. The only Hermes feature that currently needs WSL2 specifically is the browser-based dashboard chat pane (it uses a POSIX PTY — classic CLI and gateway both run natively).
After installation:
+4
View File
@@ -13,6 +13,10 @@ Usage::
hermes-acp
"""
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
import hermes_bootstrap # noqa: F401
import asyncio
import logging
import sys
+1 -1
View File
@@ -1607,7 +1607,7 @@ def _run_llm_review(prompt: str) -> Dict[str, Any]:
# terminal. The background-thread runner also hides it; this
# belt-and-suspenders path matters when a caller invokes
# run_curator_review(synchronous=True) from the CLI.
with open(os.devnull, "w") as _devnull, \
with open(os.devnull, "w", encoding="utf-8") as _devnull, \
contextlib.redirect_stdout(_devnull), \
contextlib.redirect_stderr(_devnull):
conv_result = review_agent.run_conversation(user_message=prompt)
+3 -3
View File
@@ -754,7 +754,7 @@ def _load_context_cache() -> Dict[str, int]:
if not path.exists():
return {}
try:
with open(path) as f:
with open(path, encoding="utf-8") as f:
data = yaml.safe_load(f) or {}
return data.get("context_lengths", {})
except Exception as e:
@@ -776,7 +776,7 @@ def save_context_length(model: str, base_url: str, length: int) -> None:
path = _get_context_cache_path()
try:
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w") as f:
with open(path, "w", encoding="utf-8") as f:
yaml.dump({"context_lengths": cache}, f, default_flow_style=False)
logger.info("Cached context length %s -> %s tokens", key, f"{length:,}")
except Exception as e:
@@ -800,7 +800,7 @@ def _invalidate_cached_context_length(model: str, base_url: str) -> None:
path = _get_context_cache_path()
try:
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w") as f:
with open(path, "w", encoding="utf-8") as f:
yaml.dump({"context_lengths": cache}, f, default_flow_style=False)
except Exception as e:
logger.debug("Failed to invalidate context length cache entry %s: %s", key, e)
+1 -1
View File
@@ -144,7 +144,7 @@ def nous_rate_limit_remaining() -> Optional[float]:
"""
path = _state_path()
try:
with open(path) as f:
with open(path, encoding="utf-8") as f:
state = json.load(f)
reset_at = state.get("reset_at", 0)
remaining = reset_at - time.time()
+1 -1
View File
@@ -617,7 +617,7 @@ def _locked_update_approvals() -> Iterator[Dict[str, Any]]:
save_allowlist(data)
return
with open(lock_path, "a+") as lock_fh:
with open(lock_path, "a+", encoding="utf-8") as lock_fh:
fcntl.flock(lock_fh.fileno(), fcntl.LOCK_EX)
try:
data = load_allowlist()
+4
View File
@@ -20,6 +20,10 @@ Usage:
python batch_runner.py --dataset_file=data.jsonl --batch_size=10 --run_name=my_run --distribution=image_gen
"""
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
import hermes_bootstrap # noqa: F401
import json
import logging
import os
+85 -9
View File
@@ -9,10 +9,13 @@ Usage:
python cli.py # Start interactive mode with all tools
python cli.py --toolsets web,terminal # Start with specific toolsets
python cli.py --skills hermes-agent-dev,github-auth
python cli.py -q "your question" # Single query mode
python cli.py --list-tools # List available tools and exit
"""
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
import hermes_bootstrap # noqa: F401
import logging
import os
import shutil
@@ -728,8 +731,43 @@ def _run_cleanup():
_active_worktree: Optional[Dict[str, str]] = None
def _normalize_git_bash_path(p: Optional[str]) -> Optional[str]:
"""Translate a Git Bash-style path (``/c/Users/...``) to the native
Windows form (``C:\\Users\\...``) that Python's ``subprocess.Popen``
and ``pathlib.Path`` accept.
No-op on non-Windows and for paths that already look native. Git on
native Windows normally emits forward-slash Windows paths
(``C:/Users/...``) which both bash and Python handle, but certain
configurations (Git Bash shells, MSYS2, WSL-mounted repos) surface
``/c/...`` or ``/cygdrive/c/...`` variants.
"""
if not p:
return p
if sys.platform != "win32":
return p
import re as _re
# /c/Users/... or /C/Users/...
m = _re.match(r"^/([a-zA-Z])/(.*)$", p)
if m:
drive, rest = m.group(1), m.group(2)
return f"{drive.upper()}:\\{rest.replace('/', chr(92))}"
# /cygdrive/c/... or /mnt/c/...
m = _re.match(r"^/(?:cygdrive|mnt)/([a-zA-Z])/(.*)$", p)
if m:
drive, rest = m.group(1), m.group(2)
return f"{drive.upper()}:\\{rest.replace('/', chr(92))}"
return p
def _git_repo_root() -> Optional[str]:
"""Return the git repo root for CWD, or None if not in a repo."""
"""Return the git repo root for CWD, or None if not in a repo.
Runs through :func:`_normalize_git_bash_path` so callers can pass
the result directly to ``Path``/``subprocess.Popen(cwd=...)`` on
Windows without hitting ``C:\\c\\Users\\...`` style resolution
mistakes.
"""
import subprocess
try:
result = subprocess.run(
@@ -737,7 +775,7 @@ def _git_repo_root() -> Optional[str]:
capture_output=True, text=True, timeout=5,
)
if result.returncode == 0:
return result.stdout.strip()
return _normalize_git_bash_path(result.stdout.strip())
except Exception:
pass
return None
@@ -781,7 +819,7 @@ def _setup_worktree(repo_root: str = None) -> Optional[Dict[str, str]]:
try:
existing = gitignore.read_text() if gitignore.exists() else ""
if _ignore_entry not in existing.splitlines():
with open(gitignore, "a") as f:
with open(gitignore, "a", encoding="utf-8") as f:
if existing and not existing.endswith("\n"):
f.write("\n")
f.write(f"{_ignore_entry}\n")
@@ -832,10 +870,39 @@ def _setup_worktree(repo_root: str = None) -> Optional[Dict[str, str]]:
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(str(src), str(dst))
elif src.is_dir():
# Symlink directories (faster, saves disk)
# Symlink directories (faster, saves disk). On Windows,
# symlink creation requires Developer Mode or elevation,
# and fails with OSError otherwise — fall back to a
# recursive copy so the worktree is still usable. The
# copy is slower and uses disk, but it doesn't require
# admin and matches the Linux/macOS symlink outcome
# functionally.
if not dst.exists():
dst.parent.mkdir(parents=True, exist_ok=True)
os.symlink(str(src_resolved), str(dst))
try:
os.symlink(str(src_resolved), str(dst))
except (OSError, NotImplementedError) as _sym_err:
if sys.platform == "win32":
logger.info(
".worktreeinclude: symlink failed (%s) — "
"falling back to copytree on Windows.",
_sym_err,
)
try:
shutil.copytree(
str(src_resolved),
str(dst),
symlinks=True,
dirs_exist_ok=False,
)
except Exception as _copy_err:
logger.warning(
".worktreeinclude: copy fallback "
"also failed for %s -> %s: %s",
src, dst, _copy_err,
)
else:
raise
except Exception as e:
logger.debug("Error copying .worktreeinclude entries: %s", e)
@@ -2080,7 +2147,7 @@ def save_config_value(key_path: str, value: any) -> bool:
# Load existing config
if config_path.exists():
with open(config_path, 'r') as f:
with open(config_path, 'r', encoding="utf-8") as f:
config = yaml.safe_load(f) or {}
else:
config = {}
@@ -9706,7 +9773,7 @@ class HermesCLI:
# Debug: log to file (stdout may be devnull from redirect_stdout)
try:
_dbg = _hermes_home / "interrupt_debug.log"
with open(_dbg, "a") as _f:
with open(_dbg, "a", encoding="utf-8") as _f:
_f.write(f"{time.strftime('%H:%M:%S')} interrupt fired: msg={str(interrupt_msg)[:60]!r}, "
f"children={len(self.agent._active_children)}, "
f"parent._interrupt={self.agent._interrupt_requested}\n")
@@ -10538,7 +10605,7 @@ class HermesCLI:
# Debug: log to file when message enters interrupt queue
try:
_dbg = _hermes_home / "interrupt_debug.log"
with open(_dbg, "a") as _f:
with open(_dbg, "a", encoding="utf-8") as _f:
_f.write(f"{time.strftime('%H:%M:%S')} ENTER: queued interrupt msg={str(payload)[:60]!r}, "
f"agent_running={self._agent_running}\n")
except Exception:
@@ -12342,6 +12409,15 @@ def main(
"""
global _active_worktree
# Force UTF-8 stdio on Windows before any banner/print() runs — the
# Rich console prints Unicode box-drawing characters that would
# UnicodeEncodeError on cp1252. No-op on Linux/macOS.
try:
from hermes_cli.stdio import configure_windows_stdio
configure_windows_stdio()
except Exception:
pass
# Signal to terminal_tool that we're in interactive mode
# This enables interactive sudo password prompts with timeout
os.environ["HERMES_INTERACTIVE"] = "1"
+18 -3
View File
@@ -14,6 +14,7 @@ import contextvars
import json
import logging
import os
import shutil
import subprocess
import sys
@@ -714,7 +715,21 @@ def _run_job_script(script_path: str) -> tuple[bool, str]:
# choice explicit here keeps the allowed surface small and auditable.
suffix = path.suffix.lower()
if suffix in (".sh", ".bash"):
argv = ["/bin/bash", str(path)]
# Resolve bash dynamically so Windows (Git Bash) and Linux/macOS
# all work. On native Windows without Git for Windows installed
# shutil.which returns None — fall back to a clear error rather
# than a FileNotFoundError with a confusing "[WinError 2]"
# traceback.
_bash = shutil.which("bash") or (
"/bin/bash" if os.path.isfile("/bin/bash") else None
)
if _bash is None:
return False, (
f"Cannot run .sh/.bash script {path.name!r}: bash not found on PATH. "
"On Windows, install Git for Windows (which ships Git Bash) "
"or rewrite the script as Python (.py)."
)
argv = [_bash, str(path)]
else:
argv = [sys.executable, str(path)]
@@ -1213,7 +1228,7 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
import yaml
_cfg_path = str(_get_hermes_home() / "config.yaml")
if os.path.exists(_cfg_path):
with open(_cfg_path) as _f:
with open(_cfg_path, encoding="utf-8") as _f:
_cfg = yaml.safe_load(_f) or {}
_cfg = _expand_env_vars(_cfg)
_model_cfg = _cfg.get("model", {})
@@ -1596,7 +1611,7 @@ def tick(verbose: bool = True, adapters=None, loop=None) -> int:
# Cross-platform file locking: fcntl on Unix, msvcrt on Windows
lock_fd = None
try:
lock_fd = open(lock_file, "w")
lock_fd = open(lock_file, "w", encoding="utf-8")
if fcntl:
fcntl.flock(lock_fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
elif msvcrt:
@@ -365,7 +365,7 @@ class TerminalBench2EvalEnv(HermesAgentBaseEnv):
os.makedirs(log_dir, exist_ok=True)
run_ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
self._streaming_path = os.path.join(log_dir, f"samples_{run_ts}.jsonl")
self._streaming_file = open(self._streaming_path, "w")
self._streaming_file = open(self._streaming_path, "w", encoding="utf-8")
self._streaming_lock = __import__("threading").Lock()
print(f" Streaming results to: {self._streaming_path}")
@@ -422,7 +422,7 @@ class YCBenchEvalEnv(HermesAgentBaseEnv):
os.makedirs(log_dir, exist_ok=True)
run_ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
self._streaming_path = os.path.join(log_dir, f"samples_{run_ts}.jsonl")
self._streaming_file = open(self._streaming_path, "w")
self._streaming_file = open(self._streaming_path, "w", encoding="utf-8")
self._streaming_lock = threading.Lock()
print(f"\nYC-Bench eval matrix: {len(self.all_eval_items)} runs")
+2 -2
View File
@@ -744,7 +744,7 @@ class TelegramAdapter(BasePlatformAdapter):
return
import yaml as _yaml
with open(config_path, "r") as f:
with open(config_path, "r", encoding="utf-8") as f:
config = _yaml.safe_load(f) or {}
# Navigate to platforms.telegram.extra.dm_topics
@@ -3516,7 +3516,7 @@ class TelegramAdapter(BasePlatformAdapter):
return
import yaml as _yaml
with open(config_path, "r") as f:
with open(config_path, "r", encoding="utf-8") as f:
config = _yaml.safe_load(f) or {}
dm_topics = (
+15 -5
View File
@@ -21,6 +21,7 @@ import logging
import os
import platform
import re
import shutil
import signal
import subprocess
@@ -177,10 +178,15 @@ def check_whatsapp_requirements() -> bool:
WhatsApp requires a Node.js bridge for most implementations.
"""
# Check for Node.js
# Check for Node.js. Resolve via shutil.which so we respect PATHEXT
# (node.exe vs node) and get a meaningful "not installed" signal
# instead of spawning a cmd flash on Windows.
_node = shutil.which("node")
if not _node:
return False
try:
result = subprocess.run(
["node", "--version"],
[_node, "--version"],
capture_output=True,
text=True,
timeout=5
@@ -464,9 +470,13 @@ class WhatsAppAdapter(BasePlatformAdapter):
bridge_dir = bridge_path.parent
if not (bridge_dir / "node_modules").exists():
print(f"[{self.name}] Installing WhatsApp bridge dependencies...")
# Resolve npm path so Windows can execute the .cmd shim.
# shutil.which honours PATHEXT; on POSIX it returns the
# plain executable path.
_npm_bin = shutil.which("npm") or "npm"
try:
install_result = subprocess.run(
["npm", "install", "--silent"],
[_npm_bin, "install", "--silent"],
cwd=str(bridge_dir),
capture_output=True,
text=True,
@@ -516,7 +526,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
# messages are preserved for troubleshooting.
whatsapp_mode = os.getenv("WHATSAPP_MODE", "self-chat")
self._bridge_log = self._session_path.parent / "bridge.log"
bridge_log_fh = open(self._bridge_log, "a")
bridge_log_fh = open(self._bridge_log, "a", encoding="utf-8")
self._bridge_log_fh = bridge_log_fh
# Build bridge subprocess environment.
@@ -1160,7 +1170,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
if file_size > MAX_TEXT_INJECT_BYTES:
print(f"[{self.name}] Skipping text injection for {doc_path} ({file_size} bytes > {MAX_TEXT_INJECT_BYTES})", flush=True)
continue
content = Path(doc_path).read_text(errors="replace")
content = Path(doc_path).read_text(encoding="utf-8", errors="replace")
fname = Path(doc_path).name
# Remove the doc_<hex>_ prefix for display
display_name = fname
+123 -18
View File
@@ -13,6 +13,10 @@ Usage:
python cli.py --gateway
"""
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
import hermes_bootstrap # noqa: F401
import asyncio
import dataclasses
import inspect
@@ -2784,6 +2788,48 @@ class GatewayRunner:
return
current_pid = os.getpid()
# On Windows there's no bash/setsid chain — spawn a tiny Python
# watcher directly via sys.executable instead. The watcher polls
# current_pid, waits for our exit, then runs `hermes gateway
# restart` with detach flags so the respawn survives the CLI
# that triggered the /restart command closing its console.
if sys.platform == "win32":
import textwrap
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
cmd_argv = [*hermes_cmd, "gateway", "restart"]
watcher = textwrap.dedent(
"""
import os, subprocess, sys, time
pid = int(sys.argv[1])
cmd = sys.argv[2:]
deadline = time.monotonic() + 120
while time.monotonic() < deadline:
try:
os.kill(pid, 0)
except (ProcessLookupError, PermissionError, OSError):
break
time.sleep(0.2)
_CREATE_NEW_PROCESS_GROUP = 0x00000200
_DETACHED_PROCESS = 0x00000008
_CREATE_NO_WINDOW = 0x08000000
subprocess.Popen(
cmd,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
creationflags=_CREATE_NEW_PROCESS_GROUP | _DETACHED_PROCESS | _CREATE_NO_WINDOW,
)
"""
).strip()
subprocess.Popen(
[sys.executable, "-c", watcher, str(current_pid), *cmd_argv],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
**windows_detach_popen_kwargs(),
)
return
cmd = " ".join(shlex.quote(part) for part in hermes_cmd)
shell_cmd = (
f"while kill -0 {current_pid} 2>/dev/null; do sleep 0.2; done; "
@@ -11305,30 +11351,78 @@ class GatewayRunner:
# where systemd-run --user fails due to missing D-Bus session).
# PYTHONUNBUFFERED ensures output is flushed line-by-line so the
# gateway can stream it to the messenger in near-real-time.
hermes_cmd_str = " ".join(shlex.quote(part) for part in hermes_cmd)
update_cmd = (
f"PYTHONUNBUFFERED=1 {hermes_cmd_str} update --gateway"
f" > {shlex.quote(str(output_path))} 2>&1; "
f"status=$?; printf '%s' \"$status\" > {shlex.quote(str(exit_code_path))}"
)
# Spawn `hermes update --gateway` detached so it survives gateway restart.
# --gateway enables file-based IPC for interactive prompts (stash
# restore, config migration) so the gateway can forward them to the
# user instead of silently skipping them.
# Use setsid for portable session detach (works under system services
# where systemd-run --user fails due to missing D-Bus session).
# PYTHONUNBUFFERED ensures output is flushed line-by-line so the
# gateway can stream it to the messenger in near-real-time.
#
# Windows: no bash/setsid chain. Run `hermes update --gateway`
# directly via sys.executable; redirect stdout/stderr to the same
# output files via Popen file handles; write the exit code in a
# follow-up write. A tiny Python watcher would be cleaner but
# we're already inside gateway/run.py's update path which is async,
# so the simplest correct thing is: launch an inline Python helper
# that runs the command and writes both outputs.
try:
setsid_bin = shutil.which("setsid")
if setsid_bin:
# Preferred: setsid creates a new session, fully detached
if sys.platform == "win32":
import textwrap
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
# hermes_cmd is a list of argv parts we can pass directly
# (no shell-quoting needed).
helper = textwrap.dedent(
"""
import os, subprocess, sys
output_path = sys.argv[1]
exit_code_path = sys.argv[2]
cmd = sys.argv[3:]
env = dict(os.environ)
env["PYTHONUNBUFFERED"] = "1"
with open(output_path, "wb") as f:
proc = subprocess.Popen(cmd, stdout=f, stderr=subprocess.STDOUT, env=env)
rc = proc.wait()
with open(exit_code_path, "w") as f:
f.write(str(rc))
"""
).strip()
subprocess.Popen(
[setsid_bin, "bash", "-c", update_cmd],
[
sys.executable, "-c", helper,
str(output_path), str(exit_code_path),
*hermes_cmd, "update", "--gateway",
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
start_new_session=True,
**windows_detach_popen_kwargs(),
)
else:
# Fallback: start_new_session=True calls os.setsid() in child
subprocess.Popen(
["bash", "-c", update_cmd],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
start_new_session=True,
hermes_cmd_str = " ".join(shlex.quote(part) for part in hermes_cmd)
update_cmd = (
f"PYTHONUNBUFFERED=1 {hermes_cmd_str} update --gateway"
f" > {shlex.quote(str(output_path))} 2>&1; "
f"status=$?; printf '%s' \"$status\" > {shlex.quote(str(exit_code_path))}"
)
setsid_bin = shutil.which("setsid")
if setsid_bin:
# Preferred: setsid creates a new session, fully detached
subprocess.Popen(
[setsid_bin, "bash", "-c", update_cmd],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
start_new_session=True,
)
else:
# Fallback: start_new_session=True calls os.setsid() in child
subprocess.Popen(
["bash", "-c", update_cmd],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
start_new_session=True,
)
except Exception as e:
pending_path.unlink(missing_ok=True)
exit_code_path.unlink(missing_ok=True)
@@ -15100,7 +15194,10 @@ async def start_gateway(config: Optional[GatewayConfig] = None, replace: bool =
try:
os.kill(existing_pid, 0)
time.sleep(0.5)
except (ProcessLookupError, PermissionError):
except (ProcessLookupError, PermissionError, OSError):
# OSError covers Windows' WinError 87 "invalid parameter"
# for an already-gone PID — without this the probe loop
# busy-spins for the full 10s on every --replace start.
break # Process is gone
else:
# Still alive after 10s — force kill
@@ -15385,6 +15482,14 @@ async def start_gateway(config: Optional[GatewayConfig] = None, replace: bool =
def main():
"""CLI entry point for the gateway."""
# Force UTF-8 stdio on Windows — gateway logs and startup banner would
# otherwise UnicodeEncodeError on cp1252 consoles. No-op on POSIX.
try:
from hermes_cli.stdio import configure_windows_stdio
configure_windows_stdio()
except Exception:
pass
import argparse
parser = argparse.ArgumentParser(description="Hermes Gateway - Multi-platform messaging")
+3 -3
View File
@@ -113,7 +113,7 @@ def _get_process_start_time(pid: int) -> Optional[int]:
stat_path = Path(f"/proc/{pid}/stat")
try:
# Field 22 in /proc/<pid>/stat is process start time (clock ticks).
return int(stat_path.read_text().split()[21])
return int(stat_path.read_text(encoding="utf-8").split()[21])
except (FileNotFoundError, IndexError, PermissionError, ValueError, OSError):
return None
@@ -197,7 +197,7 @@ def _read_json_file(path: Path) -> Optional[dict[str, Any]]:
if not path.exists():
return None
try:
raw = path.read_text().strip()
raw = path.read_text(encoding="utf-8").strip()
except OSError:
return None
if not raw:
@@ -523,7 +523,7 @@ def acquire_scoped_lock(scope: str, identity: str, metadata: Optional[dict[str,
try:
_proc_status = Path(f"/proc/{existing_pid}/status")
if _proc_status.exists():
for _line in _proc_status.read_text().splitlines():
for _line in _proc_status.read_text(encoding="utf-8").splitlines():
if _line.startswith("State:"):
_state = _line.split()[1]
if _state in ("T", "t"): # stopped or tracing stop
+129
View File
@@ -0,0 +1,129 @@
"""Windows UTF-8 bootstrap for Hermes entry points.
Python on Windows has two long-standing text-encoding footguns:
1. ``sys.stdout`` / ``sys.stderr`` are bound to the console code page
(``cp1252`` on US-locale installs), so ``print("café")`` crashes with
``UnicodeEncodeError: 'charmap' codec can't encode character``.
2. Child processes spawned via ``subprocess`` don't know to use UTF-8
unless ``PYTHONUTF8`` and/or ``PYTHONIOENCODING`` are set in their
environment so any Python subprocess (the execute_code sandbox,
delegation children, linter subprocesses, etc.) inherits the same
cp1252 defaults and hits the same UnicodeEncodeError.
This module fixes both on Windows *only* POSIX is untouched. It
should be imported at the very top of every Hermes entry point
(``hermes``, ``hermes-agent``, ``hermes-acp``, ``python -m gateway.run``,
``batch_runner.py``, ``cron/scheduler.py``) before any other imports
that might do file I/O or print to stdout.
What this module does on Windows:
- Sets ``os.environ["PYTHONUTF8"] = "1"`` (PEP 540 UTF-8 mode) so
every child process we spawn uses UTF-8 for ``open()`` and stdio.
- Sets ``os.environ["PYTHONIOENCODING"] = "utf-8"`` for belt-and-
suspenders some tools read this instead of / in addition to
``PYTHONUTF8``.
- Reconfigures ``sys.stdout`` / ``sys.stderr`` to UTF-8 in the current
process, using the ``reconfigure()`` API (Python 3.7+). This fixes
``print("café")`` in the parent without a re-exec.
What this module does NOT do:
- It does not re-exec Python with ``-X utf8``, so ``open()`` calls in
the *current* process still default to locale encoding. Those need
an explicit ``encoding="utf-8"`` at the call site (lint rule
``PLW1514`` / ``PYI058``). Ruff is the right tool for that sweep.
What this module does on POSIX:
- Nothing. POSIX systems are already UTF-8 by default in 99% of cases,
and we don't want to touch ``LANG``/``LC_*`` behavior that users may
have configured intentionally. If someone hits a C/POSIX locale on
Linux, they can export ``PYTHONUTF8=1`` themselves we won't override.
Idempotent: safe to call multiple times. ``_bootstrap_once`` guards
against double-reconfigure.
"""
from __future__ import annotations
import os
import sys
_IS_WINDOWS = sys.platform == "win32"
_bootstrap_applied = False
def apply_windows_utf8_bootstrap() -> bool:
"""Apply the Windows UTF-8 bootstrap if we're on Windows.
Returns True if bootstrap was applied (i.e. we're on Windows and
haven't already done this), False otherwise. The return value is
advisory callers normally don't need it, but tests may want to
assert the path was taken.
Idempotent: subsequent calls after the first are a no-op.
"""
global _bootstrap_applied
if not _IS_WINDOWS:
return False
if _bootstrap_applied:
return False
# 1. Child processes inherit these and run in UTF-8 mode.
# We use setdefault() rather than overwriting so the user can
# explicitly opt out by setting PYTHONUTF8=0 in their environment
# (or PYTHONIOENCODING=something-else) if they really want to.
os.environ.setdefault("PYTHONUTF8", "1")
os.environ.setdefault("PYTHONIOENCODING", "utf-8")
# 2. Reconfigure the current process's stdio to UTF-8. Needed
# because os.environ changes don't retroactively rebind sys.stdout
# — those were bound at interpreter startup based on the console
# code page. ``reconfigure`` is a TextIOWrapper method since 3.7.
#
# errors="replace" means that if we ever *read* something from
# stdin that isn't UTF-8 (unlikely but possible with piped input
# from legacy tools), we'll get U+FFFD replacement chars rather
# than a crash. Output is pure UTF-8.
for stream_name in ("stdout", "stderr"):
stream = getattr(sys, stream_name, None)
if stream is None:
continue
reconfigure = getattr(stream, "reconfigure", None)
if reconfigure is None:
# Not a TextIOWrapper (could be redirected to a BytesIO in
# tests, or a non-standard stream in some embedded cases).
# Skip silently — the env-var fix is still in effect for
# child processes, which is the bigger win.
continue
try:
reconfigure(encoding="utf-8", errors="replace")
except (OSError, ValueError):
# Already closed, or someone replaced it with something
# non-reconfigurable. Non-fatal.
pass
# stdin is reconfigured separately with errors="replace" too — input
# from a legacy pipe shouldn't crash the process.
stdin = getattr(sys, "stdin", None)
if stdin is not None:
reconfigure = getattr(stdin, "reconfigure", None)
if reconfigure is not None:
try:
reconfigure(encoding="utf-8", errors="replace")
except (OSError, ValueError):
pass
_bootstrap_applied = True
return True
# Apply on import — entry points just need ``import hermes_bootstrap``
# (or ``from hermes_bootstrap import apply_windows_utf8_bootstrap``) at
# the very top of their module, before importing anything else. The
# import side effect does the right thing.
apply_windows_utf8_bootstrap()
+175
View File
@@ -0,0 +1,175 @@
"""Windows subprocess compatibility helpers.
Hermes is developed on Linux / macOS and tested natively on Windows too.
Several common subprocess patterns break silently-or-loudly on Windows:
* ``["npm", "install", ...]`` on Windows ``npm`` is ``npm.cmd``, a batch
shim. ``subprocess.Popen(["npm", ...])`` fails with WinError 193
("not a valid Win32 application") because CreateProcessW can't run a
``.cmd`` file without ``shell=True`` or PATHEXT resolution.
* ``start_new_session=True`` on POSIX, this maps to ``os.setsid()`` and
actually detaches the child. On Windows it's silently ignored; the
Windows equivalent is ``CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS``
creationflags, which Python only applies when you pass them explicitly.
* Console-window flashes every ``subprocess.Popen`` of a ``.exe`` on
Windows spawns a cmd window briefly unless ``CREATE_NO_WINDOW`` is
passed. Cosmetic but jarring for background daemons.
This module centralizes the platform-branching logic so the rest of the
codebase doesn't sprinkle ``if sys.platform == "win32":`` everywhere.
**All helpers are no-ops on non-Windows** calling them in Linux/macOS
code paths is safe by design. That's the "do no damage on POSIX"
guarantee.
"""
from __future__ import annotations
import os
import shutil
import subprocess
import sys
from typing import Optional, Sequence
__all__ = [
"IS_WINDOWS",
"resolve_node_command",
"windows_detach_flags",
"windows_hide_flags",
"windows_detach_popen_kwargs",
]
IS_WINDOWS = sys.platform == "win32"
# -----------------------------------------------------------------------------
# Node ecosystem launcher resolution
# -----------------------------------------------------------------------------
def resolve_node_command(name: str, argv: Sequence[str]) -> list[str]:
"""Resolve a Node-ecosystem command name to an absolute-path argv.
On Windows, commands like ``npm``, ``npx``, ``yarn``, ``pnpm``,
``playwright``, ``prettier`` ship as ``.cmd`` files (batch shims).
``subprocess.Popen(["npm", "install"])`` fails with WinError 193
because CreateProcessW doesn't execute batch files directly.
``shutil.which(name)`` *does* resolve ``.cmd`` via PATHEXT and returns
the fully-qualified path which CreateProcessW accepts because the
extension tells Windows to route through ``cmd.exe /c``.
On POSIX ``shutil.which`` also returns a fully-qualified path when
found. That's a small change from bare-name resolution (the OS does
its own PATH search) but functionally identical and has the side
benefit of making the argv reproducible in logs.
Behavior when the command is not on PATH:
- On Windows: return the bare name caller can still try with
``shell=True`` as a last resort, OR the subsequent Popen will
raise FileNotFoundError with a readable error we want to surface.
- On POSIX: same. Bare ``npm`` on a Linux box without npm installed
fails the same way it did before this function existed.
Args:
name: The command name to resolve (``npm``, ``npx``, ``node`` ).
argv: The remaining arguments. Must NOT include ``name`` itself
this function builds the full argv list.
Returns:
A list suitable for passing to subprocess.Popen/run/call.
"""
resolved = shutil.which(name)
if resolved:
return [resolved, *argv]
return [name, *argv]
# -----------------------------------------------------------------------------
# Detached / hidden process creation
# -----------------------------------------------------------------------------
# Win32 CreationFlags — defined here rather than imported from subprocess
# because CREATE_NO_WINDOW and DETACHED_PROCESS aren't guaranteed to be
# present on stdlib subprocess on older Pythons or non-Windows builds.
_CREATE_NEW_PROCESS_GROUP = 0x00000200
_DETACHED_PROCESS = 0x00000008
_CREATE_NO_WINDOW = 0x08000000
def windows_detach_flags() -> int:
"""Return Win32 creationflags that detach a child from the parent
console and process group. 0 on non-Windows.
Pair with ``start_new_session=False`` (default) when calling
subprocess.Popen on POSIX use ``start_new_session=True`` instead,
which maps to ``os.setsid()`` in the child.
Rationale:
- ``CREATE_NEW_PROCESS_GROUP`` child has its own process group so
Ctrl+C in the parent console doesn't propagate.
- ``DETACHED_PROCESS`` child has no console at all. Necessary for
background daemons (gateway watchers, update respawners) because
without it, closing the console kills the child.
- ``CREATE_NO_WINDOW`` suppress the brief cmd flash that would
otherwise appear when launching a console app. Redundant with
DETACHED_PROCESS but explicit for clarity.
"""
if not IS_WINDOWS:
return 0
return _CREATE_NEW_PROCESS_GROUP | _DETACHED_PROCESS | _CREATE_NO_WINDOW
def windows_hide_flags() -> int:
"""Return Win32 creationflags that merely hide the child's console
window without detaching the child. 0 on non-Windows.
Use for short-lived console apps spawned as part of a larger
operation (``taskkill``, ``where``, version probes) where we want no
flash but also want to collect stdout/exit code synchronously.
The key difference from :func:`windows_detach_flags`: NO
``DETACHED_PROCESS`` the child still inherits stdio handles so
``capture_output=True`` works. ``DETACHED_PROCESS`` would sever
stdio and break stdout capture.
"""
if not IS_WINDOWS:
return 0
return _CREATE_NO_WINDOW
def windows_detach_popen_kwargs() -> dict:
"""Return a dict of Popen kwargs that detach a child on Windows and
fall back to the POSIX equivalent (``start_new_session=True``) on
Linux/macOS.
Usage pattern:
.. code-block:: python
subprocess.Popen(
argv,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
stdin=subprocess.DEVNULL,
close_fds=True,
**windows_detach_popen_kwargs(),
)
This replaces the unsafe-on-Windows pattern:
.. code-block:: python
subprocess.Popen(..., start_new_session=True)
which silently fails to detach on Windows (the flag is accepted but
has no effect the child stays attached to the parent's console
and dies when the console closes).
"""
if IS_WINDOWS:
return {"creationflags": windows_detach_flags()}
return {"start_new_session": True}
+3 -3
View File
@@ -573,7 +573,7 @@ def create_quick_snapshot(
"total_size": sum(manifest.values()),
"files": manifest,
}
with open(snap_dir / "manifest.json", "w") as f:
with open(snap_dir / "manifest.json", "w", encoding="utf-8") as f:
json.dump(meta, f, indent=2)
# Auto-prune
@@ -599,7 +599,7 @@ def list_quick_snapshots(
manifest_path = d / "manifest.json"
if manifest_path.exists():
try:
with open(manifest_path) as f:
with open(manifest_path, encoding="utf-8") as f:
results.append(json.load(f))
except (json.JSONDecodeError, OSError):
results.append({"id": d.name, "file_count": 0, "total_size": 0})
@@ -629,7 +629,7 @@ def restore_quick_snapshot(
if not manifest_path.exists():
return False
with open(manifest_path) as f:
with open(manifest_path, encoding="utf-8") as f:
meta = json.load(f)
restored = 0
+15 -7
View File
@@ -212,7 +212,7 @@ def get_container_exec_info() -> Optional[dict]:
try:
info = {}
with open(container_mode_file, "r") as f:
with open(container_mode_file, "r", encoding="utf-8") as f:
for line in f:
line = line.strip()
if "=" in line and not line.startswith("#"):
@@ -297,7 +297,7 @@ def _is_container() -> bool:
return True
# LXC / cgroup-based detection
try:
with open("/proc/1/cgroup", "r") as f:
with open("/proc/1/cgroup", "r", encoding="utf-8") as f:
cgroup_content = f.read()
if "docker" in cgroup_content or "lxc" in cgroup_content or "kubepods" in cgroup_content:
return True
@@ -3452,7 +3452,7 @@ def migrate_config(interactive: bool = True, quiet: bool = False) -> Dict[str, A
if not manifest_file.exists():
continue
try:
with open(manifest_file) as _mf:
with open(manifest_file, encoding="utf-8") as _mf:
manifest = yaml.safe_load(_mf) or {}
except Exception:
manifest = {}
@@ -4696,11 +4696,19 @@ def edit_config():
# Find editor
editor = os.getenv('EDITOR') or os.getenv('VISUAL')
if not editor:
# Try common editors
for cmd in ['nano', 'vim', 'vi', 'code', 'notepad']:
import shutil
# Try common editors — order is platform-aware so Windows users
# land on a working editor (notepad) even without Git Bash or nano
# installed. On POSIX, prefer nano/vim over code/notepad because
# it's more likely to be present on headless / server systems.
import shutil
import sys as _sys
if _sys.platform == "win32":
candidates = ['notepad', 'code', 'vim', 'vi', 'nano']
else:
candidates = ['nano', 'vim', 'vi', 'code', 'notepad']
for cmd in candidates:
if shutil.which(cmd):
editor = cmd
break
+7 -4
View File
@@ -598,7 +598,7 @@ def run_doctor(args):
# Detect stale root-level model keys (known bug source — PR #4329)
try:
import yaml
with open(config_path) as f:
with open(config_path, encoding="utf-8") as f:
raw_config = yaml.safe_load(f) or {}
stale_root_keys = [k for k in ("provider", "base_url") if k in raw_config and isinstance(raw_config[k], str)]
if stale_root_keys:
@@ -1059,7 +1059,8 @@ def run_doctor(args):
check_warn("Node.js not found", "(optional, needed for browser tools)")
# npm audit for all Node.js packages
if _safe_which("npm"):
_npm_bin = _safe_which("npm")
if _npm_bin:
npm_dirs = [
(PROJECT_ROOT, "Browser tools (agent-browser)"),
(PROJECT_ROOT / "scripts" / "whatsapp-bridge", "WhatsApp bridge"),
@@ -1068,8 +1069,10 @@ def run_doctor(args):
if not (npm_dir / "node_modules").exists():
continue
try:
# Use resolved absolute path so Windows can execute
# npm.cmd (CreateProcessW can't run bare .cmd names).
audit_result = subprocess.run(
["npm", "audit", "--json"],
[_npm_bin, "audit", "--json"],
cwd=str(npm_dir),
capture_output=True, text=True, timeout=30,
)
@@ -1396,7 +1399,7 @@ def run_doctor(args):
import yaml as _yaml
_mem_cfg_path = HERMES_HOME / "config.yaml"
if _mem_cfg_path.exists():
with open(_mem_cfg_path) as _f:
with open(_mem_cfg_path, encoding="utf-8") as _f:
_raw_cfg = _yaml.safe_load(_f) or {}
_active_memory_provider = (_raw_cfg.get("memory") or {}).get("provider", "")
except Exception:
+49 -8
View File
@@ -232,6 +232,10 @@ def _graceful_restart_via_sigusr1(pid: int, drain_timeout: float) -> bool:
# Process still exists but we can't signal it. Treat as alive
# so the caller falls back.
pass
except OSError:
# Windows raises OSError (WinError 87 "invalid parameter") for
# a gone PID — treat the same as ProcessLookupError.
return True
_time.sleep(0.5)
# Drain didn't finish in time.
return False
@@ -441,6 +445,25 @@ def launch_detached_profile_gateway_restart(profile: str, old_pid: int) -> bool:
if old_pid <= 0:
return False
# The watcher is a tiny Python subprocess that polls the old PID and
# respawns the gateway once it's gone. Both legs of the chain need
# platform-appropriate detach semantics:
#
# POSIX — ``start_new_session=True`` (os.setsid in the child) detaches
# from the parent's process group so Ctrl+C in the CLI doesn't
# propagate and the watcher/gateway survive the CLI exiting.
#
# Windows — ``start_new_session`` is silently accepted but does NOT
# detach. The watcher stays attached to the CLI's console and dies
# when the user closes the terminal, leaving ``hermes update`` users
# with no running gateway until they re-invoke ``hermes gateway``
# manually. The Win32 equivalent is the ``CREATE_NEW_PROCESS_GROUP |
# DETACHED_PROCESS | CREATE_NO_WINDOW`` creationflags bundle.
#
# ``windows_detach_popen_kwargs()`` returns the right kwargs for the
# host platform and is a no-op on POSIX (just ``start_new_session=True``).
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
watcher = textwrap.dedent(
"""
import os
@@ -458,22 +481,39 @@ def launch_detached_profile_gateway_restart(profile: str, old_pid: int) -> bool:
break
except PermissionError:
pass
except OSError:
# Windows: gone PID raises OSError (WinError 87).
break
time.sleep(0.2)
subprocess.Popen(
cmd,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
start_new_session=True,
)
# Platform-appropriate detach for the respawned gateway. On POSIX
# start_new_session=True maps to os.setsid; on Windows we need
# explicit creationflags because start_new_session is a no-op there.
_popen_kwargs = {
"stdout": subprocess.DEVNULL,
"stderr": subprocess.DEVNULL,
}
if sys.platform == "win32":
_CREATE_NEW_PROCESS_GROUP = 0x00000200
_DETACHED_PROCESS = 0x00000008
_CREATE_NO_WINDOW = 0x08000000
_popen_kwargs["creationflags"] = (
_CREATE_NEW_PROCESS_GROUP | _DETACHED_PROCESS | _CREATE_NO_WINDOW
)
else:
_popen_kwargs["start_new_session"] = True
subprocess.Popen(cmd, **_popen_kwargs)
"""
).strip()
try:
# Same platform-aware detach for the watcher process itself — so
# closing the user's terminal doesn't kill the watcher.
subprocess.Popen(
[sys.executable, "-c", watcher, str(old_pid), *_gateway_run_args_for_profile(profile)],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
start_new_session=True,
**windows_detach_popen_kwargs(),
)
except OSError:
return False
@@ -935,7 +975,8 @@ def stop_profile_gateway() -> bool:
try:
os.kill(pid, 0)
_time.sleep(0.5)
except (ProcessLookupError, PermissionError):
except (ProcessLookupError, PermissionError, OSError):
# OSError covers Windows' WinError 87 for gone PIDs.
break
if get_running_pid() is None:
+1 -1
View File
@@ -205,7 +205,7 @@ def _cmd_test(args) -> None:
if getattr(args, "payload_file", None):
try:
custom = json.loads(Path(args.payload_file).read_text())
custom = json.loads(Path(args.payload_file).read_text(encoding="utf-8"))
if isinstance(custom, dict):
payload.update(custom)
else:
+26 -14
View File
@@ -2835,7 +2835,7 @@ def _pid_alive(pid: Optional[int]) -> bool:
# where we have a cheap, deterministic process-state probe.
if sys.platform == "linux":
try:
with open(f"/proc/{int(pid)}/status", "r") as f:
with open(f"/proc/{int(pid)}/status", "r", encoding="utf-8") as f:
for line in f:
if line.startswith("State:"):
# "State:\tZ (zombie)" → dead
@@ -2911,7 +2911,10 @@ def _terminate_reclaimed_worker(
if _pid_alive(pid):
try:
kill(int(pid), signal.SIGKILL)
# signal.SIGKILL doesn't exist on Windows; fall back to SIGTERM
# (which maps to TerminateProcess via the stdlib shim).
_sigkill = getattr(signal, "SIGKILL", signal.SIGTERM)
kill(int(pid), _sigkill)
info["sigkill"] = True
except (ProcessLookupError, OSError):
return info
@@ -3035,7 +3038,9 @@ def enforce_max_runtime(
time.sleep(0.5)
if _pid_alive(pid):
try:
kill(pid, signal.SIGKILL)
# signal.SIGKILL doesn't exist on Windows.
_sigkill = getattr(signal, "SIGKILL", signal.SIGTERM)
kill(pid, _sigkill)
killed = True
except (ProcessLookupError, OSError):
pass
@@ -3514,17 +3519,24 @@ def dispatch_once(
# cleanly without calling ``kanban_complete`` / ``kanban_block``
# (protocol violation — auto-block) from a real crash (OOM killer,
# SIGKILL, non-zero exit — existing counter behavior).
try:
while True:
try:
_pid, _status = os.waitpid(-1, os.WNOHANG)
except ChildProcessError:
break
if _pid == 0:
break
_record_worker_exit(_pid, _status)
except Exception:
pass
#
# Windows has no zombies / no os.WNOHANG — subprocess.Popen handles
# are freed when the Python object is garbage-collected or .wait() is
# called explicitly. The kanban dispatcher discards the Popen handle
# after spawn (``_default_spawn`` → abandon), so on Windows there's
# nothing to reap here — skip the whole block.
if os.name != "nt":
try:
while True:
try:
_pid, _status = os.waitpid(-1, os.WNOHANG)
except ChildProcessError:
break
if _pid == 0:
break
_record_worker_exit(_pid, _status)
except Exception:
pass
result = DispatchResult()
result.reclaimed = release_stale_claims(conn)
+19 -2
View File
@@ -43,6 +43,11 @@ Usage:
hermes claw migrate --dry-run # Preview migration without changes
"""
# IMPORTANT: hermes_bootstrap must be the very first import — it sets up
# UTF-8 stdio on Windows so print()/subprocess children don't hit
# UnicodeEncodeError with non-ASCII characters. No-op on POSIX.
import hermes_bootstrap # noqa: F401
import argparse
import json
import os
@@ -7965,10 +7970,15 @@ def _cmd_update_impl(args, gateway_mode: bool):
print(
f"{len(_stuck)} gateway process(es) ignored SIGTERM — force-killing"
)
from gateway.status import terminate_pid as _terminate_pid
for pid in _stuck:
try:
os.kill(pid, _signal.SIGKILL)
except (ProcessLookupError, PermissionError):
# Routes through taskkill /T /F on Windows,
# SIGKILL on POSIX — _signal.SIGKILL doesn't
# exist on Windows so the old raw os.kill call
# used to crash the entire update path.
_terminate_pid(pid, force=True)
except (ProcessLookupError, PermissionError, OSError):
pass
# Give the OS a beat to reap the processes so the
# watchers see them exit and respawn.
@@ -8554,6 +8564,13 @@ def _build_provider_choices() -> list[str]:
def main():
"""Main entry point for hermes CLI."""
# Force UTF-8 stdio on Windows before anything prints. No-op elsewhere.
try:
from hermes_cli.stdio import configure_windows_stdio
configure_windows_stdio()
except Exception:
pass
from hermes_cli._parser import build_top_level_parser
parser, subparsers, chat_parser = build_top_level_parser()
+2 -2
View File
@@ -69,7 +69,7 @@ def _install_dependencies(provider_name: str) -> None:
try:
import yaml
with open(yaml_path) as f:
with open(yaml_path, encoding="utf-8") as f:
meta = yaml.safe_load(f) or {}
except Exception:
return
@@ -377,7 +377,7 @@ def _write_env_vars(env_path: Path, env_writes: dict) -> None:
if key not in updated_keys:
new_lines.append(f"{key}={val}")
env_path.write_text("\n".join(new_lines) + "\n")
env_path.write_text("\n".join(new_lines) + "\n", encoding="utf-8")
# ---------------------------------------------------------------------------
+2 -2
View File
@@ -173,7 +173,7 @@ def _read_disk_cache() -> tuple[dict[str, Any] | None, float]:
except (OSError, FileNotFoundError):
return (None, 0.0)
try:
with open(path) as fh:
with open(path, encoding="utf-8") as fh:
data = json.load(fh)
except (OSError, json.JSONDecodeError):
return (None, 0.0)
@@ -187,7 +187,7 @@ def _write_disk_cache(data: dict[str, Any]) -> None:
try:
path.parent.mkdir(parents=True, exist_ok=True)
tmp = path.with_suffix(path.suffix + ".tmp")
with open(tmp, "w") as fh:
with open(tmp, "w", encoding="utf-8") as fh:
json.dump(data, fh, indent=2)
fh.write("\n")
atomic_replace(tmp, path)
+1 -1
View File
@@ -174,7 +174,7 @@ def run_oneshot(
# Redirect stderr AND stdout to devnull for the entire call tree.
# We'll print the final response to the real stdout at the end.
real_stdout = sys.stdout
devnull = open(os.devnull, "w")
devnull = open(os.devnull, "w", encoding="utf-8")
try:
with redirect_stdout(devnull), redirect_stderr(devnull):
+1 -1
View File
@@ -870,7 +870,7 @@ class PluginManager:
if yaml is None:
logger.warning("PyYAML not installed cannot load %s", manifest_file)
return None
data = yaml.safe_load(manifest_file.read_text()) or {}
data = yaml.safe_load(manifest_file.read_text(encoding="utf-8")) or {}
name = data.get("name", plugin_dir.name)
key = f"{prefix}/{plugin_dir.name}" if prefix else name
+2 -2
View File
@@ -127,7 +127,7 @@ def _read_manifest(plugin_dir: Path) -> dict:
try:
import yaml
with open(manifest_file) as f:
with open(manifest_file, encoding="utf-8") as f:
return yaml.safe_load(f) or {}
except Exception as e:
logger.warning("Failed to read plugin.yaml in %s: %s", plugin_dir, e)
@@ -703,7 +703,7 @@ def _discover_all_plugins() -> list:
description = ""
if yaml:
try:
with open(manifest_file) as f:
with open(manifest_file, encoding="utf-8") as f:
manifest = yaml.safe_load(f) or {}
name = manifest.get("name", d.name)
version = manifest.get("version", "")
+13 -6
View File
@@ -354,7 +354,7 @@ def _read_config_model(profile_dir: Path) -> tuple:
return None, None
try:
import yaml
with open(config_path, "r") as f:
with open(config_path, "r", encoding="utf-8") as f:
cfg = yaml.safe_load(f) or {}
model_cfg = cfg.get("model", {})
if isinstance(model_cfg, str):
@@ -758,7 +758,6 @@ def _cleanup_gateway_service(name: str, profile_dir: Path) -> None:
def _stop_gateway_process(profile_dir: Path) -> None:
"""Stop a running gateway process via its PID file."""
import signal as _signal
import time as _time
pid_file = profile_dir / "gateway.pid"
@@ -769,19 +768,27 @@ def _stop_gateway_process(profile_dir: Path) -> None:
raw = pid_file.read_text().strip()
data = json.loads(raw) if raw.startswith("{") else {"pid": int(raw)}
pid = int(data["pid"])
os.kill(pid, _signal.SIGTERM)
# Route through terminate_pid so Windows uses the appropriate
# primitive (taskkill / TerminateProcess) — raw os.kill with
# _signal.SIGKILL raises AttributeError at import time on Windows,
# and raw os.kill with SIGTERM doesn't cascade to child processes
# the same way taskkill /T does.
from gateway.status import terminate_pid as _terminate_pid
_terminate_pid(pid) # graceful first
# Wait up to 10s for graceful shutdown
for _ in range(20):
_time.sleep(0.5)
try:
os.kill(pid, 0)
except ProcessLookupError:
except (ProcessLookupError, OSError):
# OSError covers Windows' WinError 87 "invalid parameter"
# returned for an invalid/gone PID probe.
print(f"✓ Gateway stopped (PID {pid})")
return
# Force kill
try:
os.kill(pid, _signal.SIGKILL)
except ProcessLookupError:
_terminate_pid(pid, force=True)
except (ProcessLookupError, OSError):
pass
print(f"✓ Gateway force-stopped (PID {pid})")
except (ProcessLookupError, PermissionError):
+8 -5
View File
@@ -7,11 +7,14 @@ keystrokes can be fed back in. The only caller today is the
Design constraints:
* **POSIX-only.** Hermes Agent supports Windows exclusively via WSL, which
exposes a native POSIX PTY via ``openpty(3)``. Native Windows Python
has no PTY; :class:`PtyUnavailableError` is raised with a user-readable
install/platform message so the dashboard can render a banner instead of
crashing.
* **POSIX-only.** This module depends on ``fcntl``, ``termios``, and
``ptyprocess``, none of which exist on native Windows Python. Native
Windows ConPTY is a different API (Windows 10 build 17763+) and would
need a separate Windows implementation (``pywinpty``) that's tracked
as a future enhancement. On native Windows, importing this module
raises :class:`ImportError` and the dashboard's ``/chat`` tab shows a
WSL-recommended banner instead of crashing. Every other feature in the
dashboard (sessions, jobs, metrics, config editor) works natively.
* **Zero Node dependency on the server side.** We use :mod:`ptyprocess`,
which is a pure-Python wrapper around the OS calls. The browser talks
to the same ``hermes --tui`` binary it would launch from the CLI, so
+60 -4
View File
@@ -84,18 +84,34 @@ def resolve_hermes_bin() -> Optional[str]:
1. ``sys.argv[0]`` if it resolves to a real executable.
2. ``shutil.which("hermes")`` on PATH.
3. ``None`` caller should fall back to ``python -m hermes_cli.main``.
Windows note: ``os.access(path, os.X_OK)`` returns True for ``.py`` and
``.pyc`` files on Windows (the OS treats anything listed in PATHEXT as
executable, and Python files are often registered there). But
``subprocess.run([script.py, ...])`` can't actually execute a .py
directly CreateProcessW needs a real .exe, not a script associated
with the Python launcher. On Windows we therefore skip the argv[0]
fast-path when it points at a .py file and fall through to either
``hermes.exe`` on PATH or the ``sys.executable -m hermes_cli.main``
fallback.
"""
argv0 = sys.argv[0]
_is_windows = sys.platform == "win32"
def _is_python_script(p: str) -> bool:
return p.lower().endswith((".py", ".pyc"))
# Absolute path to an executable (covers nix store, venv wrappers, etc.)
if os.path.isabs(argv0) and os.path.isfile(argv0) and os.access(argv0, os.X_OK):
return argv0
if not (_is_windows and _is_python_script(argv0)):
return argv0
# Relative path — resolve against CWD
if not argv0.startswith("-") and os.path.isfile(argv0):
abs_path = os.path.abspath(argv0)
if os.access(abs_path, os.X_OK):
return abs_path
if not (_is_windows and _is_python_script(abs_path)):
return abs_path
# PATH lookup
path_bin = shutil.which("hermes")
@@ -142,8 +158,48 @@ def relaunch(
preserve_inherited: bool = True,
original_argv: Optional[Sequence[str]] = None,
) -> None:
"""Replace the current process with a fresh hermes invocation."""
"""Replace the current process with a fresh hermes invocation.
On POSIX we use ``os.execvp`` which replaces the running process with
the new one in place same PID, no double-fork. That's what the
relaunch contract wants: "run hermes again as if the user had typed
the new argv".
Windows has no native exec semantics ``os.execvp`` on Windows
*emulates* exec by spawning the child and exiting the parent, but
only works when the target is a real Win32 executable. Our target
is usually ``hermes.exe`` (a Python console-script shim that wraps
``python -m hermes_cli.main``) or a ``.cmd`` batch file, and both
raise ``OSError(8, "Exec format error")`` on Windows' execvp.
The Windows-correct pattern is: spawn the child with ``subprocess.run``
(which routes through ``cmd.exe`` via ``shell=False`` + PATHEXT resolution),
wait for it to exit, then propagate its exit code via ``sys.exit``.
That's functionally equivalent — the user sees "hermes exited, then
new hermes started" — just with two PIDs in play instead of one.
"""
new_argv = build_relaunch_argv(
extra_args, preserve_inherited=preserve_inherited, original_argv=original_argv
)
os.execvp(new_argv[0], new_argv)
if sys.platform == "win32":
# Windows: subprocess + exit, because execvp can't swap to .cmd/.exe shims.
import subprocess
try:
result = subprocess.run(new_argv)
sys.exit(result.returncode)
except KeyboardInterrupt:
sys.exit(130)
except OSError as exc:
# Surface a helpful error rather than the raw OSError — the
# caller used to see ``[Errno 8] Exec format error`` which is
# cryptic. Common causes: ``hermes`` not on PATH yet (install
# hasn't propagated User PATH into this shell) or a stale shim.
print(
f"\nHermes relaunch failed: {exc}\n"
f"Command: {' '.join(new_argv)}\n"
f"Fix: open a new terminal so PATH picks up, then re-run hermes.",
file=sys.stderr,
)
sys.exit(1)
else:
os.execvp(new_argv[0], new_argv)
+2 -2
View File
@@ -1257,7 +1257,7 @@ def do_snapshot_export(output_path: str, console: Optional[Console] = None) -> N
sys.stdout.write(payload)
else:
out = Path(output_path)
out.write_text(payload)
out.write_text(payload, encoding="utf-8")
c.print(f"[bold green]Snapshot exported:[/] {out}")
c.print(f"[dim]{len(installed)} skill(s), {len(tap_list)} tap(s)[/]\n")
@@ -1274,7 +1274,7 @@ def do_snapshot_import(input_path: str, force: bool = False,
return
try:
snapshot = json.loads(inp.read_text())
snapshot = json.loads(inp.read_text(encoding="utf-8"))
except json.JSONDecodeError:
c.print(f"[bold red]Error:[/] Invalid JSON in {inp}\n")
return
+252
View File
@@ -0,0 +1,252 @@
"""Windows-safe stdio configuration.
On Windows, Python's ``sys.stdout``/``sys.stderr`` default to the console's
active code page (often ``cp1252``, sometimes ``cp437``, occasionally ``cp932``
on Japanese locales, etc.). Hermes's banners, tool output feed, and slash
command listings all contain Unicode: box-drawing characters (````),
mathematical and geometric symbols (`` ``), and user-supplied
text in any language. Printing those to a cp1252 console raises
``UnicodeEncodeError: 'charmap' codec can't encode character…`` and kills the
whole CLI before the REPL even opens.
The fix is to force UTF-8 on the Python side and also flip the console's
code page to UTF-8 (65001). Both matter: Python-level only helps when
Python's stdout is a real TTY; code-page flipping lets subprocesses and
child Python ``print()`` calls agree on encoding.
This module is a no-op on every non-Windows platform, and idempotent.
Entry points (``cli.py`` ``main``, ``hermes_cli/main.py`` CLI dispatch,
``gateway/run.py`` startup) call :func:`configure_windows_stdio` exactly
once early in startup.
Patterns cribbed from Claude Code (``src/utils/platform.ts``), OpenCode
(``packages/opencode/src/pty/index.ts`` env injection), and OpenAI Codex
(``codex-rs/core/src/unified_exec/process_manager.rs``). None of those
actually flip the console code page they rely on their runtime (Node or
Rust) writing UTF-16 to the Win32 console API and letting the terminal
sort it out. Python doesn't get that luxury.
"""
from __future__ import annotations
import os
import sys
__all__ = ["configure_windows_stdio", "is_windows"]
_CONFIGURED = False
def is_windows() -> bool:
"""Return True iff running on native Windows (not WSL)."""
return sys.platform == "win32"
def _flip_console_code_page_to_utf8() -> None:
"""Set the attached console's input and output code pages to UTF-8.
Uses ``SetConsoleCP`` / ``SetConsoleOutputCP`` via ``ctypes``. Failure
is silent if there's no attached console (e.g. Hermes is running
behind a redirected stdout, under a service, or inside a PTY-less CI
runner) these calls simply return 0 and we move on.
CP_UTF8 is 65001.
"""
try:
import ctypes
kernel32 = ctypes.windll.kernel32 # type: ignore[attr-defined]
# Best-effort; if there's no console attached these just fail silently.
kernel32.SetConsoleCP(65001)
kernel32.SetConsoleOutputCP(65001)
except Exception:
# ctypes import, missing kernel32, or non-Windows — any failure here
# is non-fatal. We've still reconfigured Python's own streams below.
pass
def _reconfigure_stream(stream, *, encoding: str = "utf-8", errors: str = "replace") -> None:
"""Reconfigure a text stream to UTF-8 in place.
Uses ``TextIOWrapper.reconfigure`` (Python 3.7+). If the stream isn't
a ``TextIOWrapper`` (e.g. it's been redirected to an ``io.StringIO``
during tests), we skip rather than blow up.
"""
try:
reconfigure = getattr(stream, "reconfigure", None)
if reconfigure is None:
return
reconfigure(encoding=encoding, errors=errors)
except Exception:
pass
def configure_windows_stdio() -> bool:
"""Force UTF-8 stdio on Windows. No-op elsewhere.
Idempotent safe to call multiple times from different entry points.
Returns ``True`` if anything was actually changed, ``False`` on
non-Windows or on a repeat call.
Set ``HERMES_DISABLE_WINDOWS_UTF8=1`` in the environment to opt out
(for diagnosing encoding-related bugs by forcing the old cp1252 path).
Also sets a sensible default ``EDITOR`` on Windows if none is already
set see :func:`_default_windows_editor`.
"""
global _CONFIGURED
if _CONFIGURED:
return False
if not is_windows():
# Mark configured so repeated calls on POSIX are true no-ops.
_CONFIGURED = True
return False
if os.environ.get("HERMES_DISABLE_WINDOWS_UTF8") in ("1", "true", "True", "yes"):
_CONFIGURED = True
return False
# Encourage every child Python process spawned by the agent to also use
# UTF-8 for its stdio. PYTHONIOENCODING wins over the locale-based
# default in subprocesses. Don't override an explicit user setting.
os.environ.setdefault("PYTHONIOENCODING", "utf-8")
# PYTHONUTF8 = 1 enables UTF-8 Mode globally for any Python subprocess
# (PEP 540). Again, don't override an explicit setting.
os.environ.setdefault("PYTHONUTF8", "1")
# Set EDITOR to a working Windows default if neither EDITOR nor VISUAL
# is set. prompt_toolkit's ``open_in_editor`` falls back to POSIX-only
# paths (``/usr/bin/nano``, ``/usr/bin/vi``) that don't exist on
# Windows — Ctrl+X Ctrl+E and ``/edit`` silently do nothing there
# otherwise. This happens even with full Git for Windows installed,
# so it's not a MinGit-specific issue.
_default_editor = _default_windows_editor()
if _default_editor and not os.environ.get("EDITOR") and not os.environ.get("VISUAL"):
os.environ["EDITOR"] = _default_editor
# Augment PATH with the Hermes-managed Git install directories so
# subprocess calls (bash, rg, grep, etc.) resolve even in sessions
# that started before the User PATH broadcast reached them. When
# install.ps1 adds these to User PATH via SetEnvironmentVariable,
# already-running shells don't see the change — which means hermes
# launched from the install session won't find rg / bash / grep
# even though they're "installed". Prepending the known paths here
# closes that gap. No-op when the paths don't exist (e.g. system-Git
# install without Hermes-managed PortableGit).
_augment_path_with_known_tools()
# Flip the console code page first so that any subprocess that
# inherits the console (e.g. a launched shell) also sees CP_UTF8.
_flip_console_code_page_to_utf8()
# Reconfigure Python's own stdio wrappers so ``print()`` calls from
# this process round-trip emoji / box-drawing / non-Latin text.
# ``errors="replace"`` means a genuinely unencodable byte sequence
# gets a ``?`` rather than crashing the interpreter — we prefer
# degraded output over a stack trace.
_reconfigure_stream(sys.stdout)
_reconfigure_stream(sys.stderr)
# stdin is re-configured for completeness; Hermes's interactive
# input path uses prompt_toolkit which manages its own encoding,
# but batch/pipe input benefits from UTF-8 decoding on stdin too.
_reconfigure_stream(sys.stdin)
_CONFIGURED = True
return True
def _default_windows_editor() -> str:
"""Return a Windows-appropriate default for ``$EDITOR``.
Priority order, first match wins:
1. ``notepad`` ships with every Windows install, no deps, works as a
blocking editor (``subprocess.call(["notepad", file])`` blocks until
the user closes the window). This is the "always-works" default.
The prompt_toolkit buffer's ``open_in_editor`` and Hermes's
``hermes config edit`` both honour ``$EDITOR``. Users who prefer a
different editor can override:
- VSCode: ``$env:EDITOR = "code --wait"`` (``--wait`` is critical;
without it the editor returns immediately and any input is lost)
- Notepad++: ``$env:EDITOR = "'C:\\Program Files\\Notepad++\\notepad++.exe' -multiInst -nosession"``
- Neovim: ``$env:EDITOR = "nvim"`` (if installed)
Set this before launching Hermes (User env var in Windows Settings, or
export in a PowerShell profile) and Hermes picks it up automatically.
"""
import shutil
# notepad.exe is always in %SystemRoot%\System32 on Windows, so shutil.which
# will reliably find it. Return the bare name so prompt_toolkit's shlex
# split doesn't trip over a path containing spaces.
if shutil.which("notepad"):
return "notepad"
# On the extreme off-chance notepad is missing (WinPE, Nano Server), fall
# back to nothing and let prompt_toolkit's silent no-op do its thing.
return ""
def _augment_path_with_known_tools() -> None:
"""Prepend well-known Hermes-managed tool directories to os.environ['PATH'].
Fixes the "User PATH was just updated but my process can't see it" gap on
Windows. When install.ps1 runs, it adds entries like
``%LOCALAPPDATA%\\hermes\\git\\bin`` to the User PATH via
``SetEnvironmentVariable(..., "User")``. That write propagates to newly
*spawned* processes only already-running shells (including the one the
user invokes ``hermes`` from right after install) retain their old PATH.
Any subprocess Hermes spawns bash, ``rg``, ``grep``, ``npm`` inherits
that stale PATH and reports commands as missing even though they're on
disk. Symptom: ``search_files`` reports "rg/find not available" when
the user clearly just installed ripgrep.
Patch-up strategy: add the known Hermes-managed tool directories to our
PATH at startup so subprocess calls resolve correctly. No-op on POSIX
and when the directories don't exist. The User PATH broadcast still
happens in the background for future shells; this just smooths over
the first-launch gap.
"""
if not is_windows():
return
import shutil as _shutil
local_appdata = os.environ.get("LOCALAPPDATA", "")
if not local_appdata:
return
# Known tool dirs installed by scripts/install.ps1. Kept in sync with
# the PATH entries that installer adds to User scope — the two lists
# should match so this prefill fully mirrors what a fresh shell would
# see on next launch.
candidate_dirs = [
os.path.join(local_appdata, "hermes", "git", "cmd"),
os.path.join(local_appdata, "hermes", "git", "bin"),
os.path.join(local_appdata, "hermes", "git", "usr", "bin"),
# Hermes venv Scripts directory — host of the hermes.exe shim itself,
# also where any pip-installed console scripts land. Usually already
# on PATH when the user invokes hermes, but harmless to include.
os.path.join(local_appdata, "hermes", "hermes-agent", "venv", "Scripts"),
# WinGet packages directory — where ``winget install`` drops CLI
# shims by default (ripgrep lands here as rg.exe). Covers the case
# of a system-Git install + ripgrep-via-winget that isn't yet on
# the spawning shell's PATH.
os.path.join(local_appdata, "Microsoft", "WinGet", "Links"),
]
existing = os.environ.get("PATH", "")
existing_lower = {p.lower() for p in existing.split(os.pathsep) if p}
prepend = []
for d in candidate_dirs:
if os.path.isdir(d) and d.lower() not in existing_lower:
prepend.append(d)
if prepend:
os.environ["PATH"] = os.pathsep.join([*prepend, existing])
+9 -3
View File
@@ -509,8 +509,12 @@ def _run_post_setup(post_setup_key: str):
if not node_modules.exists() and npm_bin:
_print_info(" Installing Node.js dependencies for browser tools...")
import subprocess
# Use the resolved npm_bin absolute path so subprocess.Popen can
# execute npm.cmd on Windows (CreateProcessW otherwise rejects
# batch shims). On POSIX npm_bin is the plain path — same
# behaviour as before.
result = subprocess.run(
["npm", "install", "--silent"],
[npm_bin, "install", "--silent"],
capture_output=True, text=True, cwd=str(PROJECT_ROOT)
)
if result.returncode == 0:
@@ -609,11 +613,13 @@ def _run_post_setup(post_setup_key: str):
elif post_setup_key == "camofox":
camofox_dir = PROJECT_ROOT / "node_modules" / "@askjo" / "camofox-browser"
if not camofox_dir.exists() and shutil.which("npm"):
_npm_bin = shutil.which("npm")
if not camofox_dir.exists() and _npm_bin:
_print_info(" Installing Camofox browser server...")
import subprocess
# Absolute npm path so .cmd shim executes on Windows.
result = subprocess.run(
["npm", "install", "--silent"],
[_npm_bin, "install", "--silent"],
capture_output=True, text=True, cwd=str(PROJECT_ROOT)
)
if result.returncode == 0:
+27 -2
View File
@@ -692,7 +692,7 @@ def _tail_lines(path: Path, n: int) -> List[str]:
if not path.exists():
return []
try:
text = path.read_text(errors="replace")
text = path.read_text(encoding="utf-8", errors="replace")
except OSError:
return []
lines = text.splitlines()
@@ -2979,7 +2979,20 @@ async def get_models_analytics(days: int = 30):
import re
import asyncio
from hermes_cli.pty_bridge import PtyBridge, PtyUnavailableError
# PTY bridge is POSIX-only (depends on fcntl/termios/ptyprocess). On native
# Windows the import raises; catch and leave PtyBridge=None so the rest of
# the dashboard (sessions, jobs, metrics, config editor) still loads and the
# /api/pty endpoint cleanly refuses with a WSL-suggested message.
try:
from hermes_cli.pty_bridge import PtyBridge, PtyUnavailableError
_PTY_BRIDGE_AVAILABLE = True
except ImportError as _pty_import_err: # pragma: no cover - Windows-only path
PtyBridge = None # type: ignore[assignment]
_PTY_BRIDGE_AVAILABLE = False
class PtyUnavailableError(RuntimeError): # type: ignore[no-redef]
"""Stub on platforms where pty_bridge can't be imported."""
pass
_RESIZE_RE = re.compile(rb"\x1b\[RESIZE:(\d+);(\d+)\]")
_PTY_READ_CHUNK_TIMEOUT = 0.2
@@ -3113,6 +3126,18 @@ async def pty_ws(ws: WebSocket) -> None:
await ws.accept()
# On native Windows, the POSIX PTY bridge can't be imported. Tell the
# client and close cleanly rather than pretending the feature works.
if not _PTY_BRIDGE_AVAILABLE:
await ws.send_text(
"\r\n\x1b[31mChat unavailable: the embedded terminal requires a "
"POSIX PTY, which native Windows Python doesn't provide.\x1b[0m\r\n"
"\x1b[33mInstall Hermes inside WSL2 to use the dashboard's /chat "
"tab — the rest of the dashboard works here.\x1b[0m\r\n"
)
await ws.close(code=1011)
return
# --- spawn PTY ------------------------------------------------------
resume = ws.query_params.get("resume") or None
channel = _channel_or_close_code(ws)
+2 -2
View File
@@ -233,7 +233,7 @@ def is_wsl() -> bool:
if _wsl_detected is not None:
return _wsl_detected
try:
with open("/proc/version", "r") as f:
with open("/proc/version", "r", encoding="utf-8") as f:
_wsl_detected = "microsoft" in f.read().lower()
except Exception:
_wsl_detected = False
@@ -260,7 +260,7 @@ def is_container() -> bool:
_container_detected = True
return True
try:
with open("/proc/1/cgroup", "r") as f:
with open("/proc/1/cgroup", "r", encoding="utf-8") as f:
cgroup = f.read()
if "docker" in cgroup or "podman" in cgroup or "/lxc/" in cgroup:
_container_detected = True
+1 -1
View File
@@ -50,7 +50,7 @@ def _resolve_timezone_name() -> str:
import yaml
config_path = get_config_path()
if config_path.exists():
with open(config_path) as f:
with open(config_path, encoding="utf-8") as f:
cfg = yaml.safe_load(f) or {}
tz_cfg = cfg.get("timezone", "")
if isinstance(tz_cfg, str) and tz_cfg.strip():
+26 -2
View File
@@ -36,6 +36,12 @@ dependencies = [
"edge-tts>=7.2.7,<8",
# Skills Hub (GitHub App JWT auth — optional, only needed for bot identity)
"PyJWT[crypto]>=2.12.0,<3", # CVE-2026-32597
# Windows has no IANA tzdata shipped with the OS, so Python's ``zoneinfo``
# (PEP 615) raises ``ZoneInfoNotFoundError`` for every non-UTC timezone
# out of the box. ``tzdata`` ships the Olson database as a data package
# Python resolves automatically. No-op on Linux/macOS (which have
# /usr/share/zoneinfo). Credits: PR #13182 (@sprmn24).
"tzdata>=2023.3; sys_platform == 'win32'",
]
[project.optional-dependencies]
@@ -154,7 +160,7 @@ hermes-agent = "run_agent:main"
hermes-acp = "acp_adapter.entry:main"
[tool.setuptools]
py-modules = ["run_agent", "model_tools", "toolsets", "batch_runner", "trajectory_compressor", "toolset_distributions", "cli", "hermes_constants", "hermes_state", "hermes_time", "hermes_logging", "rl_cli", "utils"]
py-modules = ["run_agent", "model_tools", "toolsets", "batch_runner", "trajectory_compressor", "toolset_distributions", "cli", "hermes_bootstrap", "hermes_constants", "hermes_state", "hermes_time", "hermes_logging", "rl_cli", "utils"]
[tool.setuptools.package-data]
hermes_cli = ["web_dist/**/*"]
@@ -182,7 +188,25 @@ exclude = ["tinker-atropos"]
[tool.ruff]
exclude = ["tinker-atropos"]
select = [] # disable all lints for now, until we've wrangled typechecks a bit more :3
preview = true # required for PLW1514 (unspecified-encoding) — preview rule
[tool.ruff.lint]
# All other lints are intentionally disabled (see comment history on this
# file) while we wrangle typechecks — but PLW1514 is too load-bearing to
# keep off. Bare open()/read_text()/write_text() in text mode defaults to
# the system locale encoding on Windows (cp1252 on US-locale installs),
# which silently corrupts any non-ASCII file content. We had three
# separate Windows sandbox regressions in one debug session before
# adding the explicit encoding. This rule keeps new code honest.
select = ["PLW1514"]
[tool.ruff.lint.per-file-ignores]
# Tests can intentionally exercise locale-encoding edge cases.
"tests/**" = ["PLW1514"]
# Skills and plugins are partially user-authored — their own conventions.
"skills/**" = ["PLW1514"]
"optional-skills/**" = ["PLW1514"]
"plugins/**" = ["PLW1514"]
[tool.uv]
exclude-newer = "7 days"
+1 -1
View File
@@ -82,7 +82,7 @@ def load_hermes_config() -> dict:
if config_path.exists():
try:
with open(config_path, "r") as f:
with open(config_path, "r", encoding='utf-8') as f:
file_config = yaml.safe_load(f) or {}
# Get model from config
+5 -1
View File
@@ -20,6 +20,10 @@ Usage:
response = agent.run_conversation("Tell me about the latest Python updates")
"""
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
import hermes_bootstrap # noqa: F401
import asyncio
import base64
import concurrent.futures
@@ -3682,7 +3686,7 @@ class AIAgent:
pass
review_agent = None
try:
with open(os.devnull, "w") as _devnull, \
with open(os.devnull, "w", encoding="utf-8") as _devnull, \
contextlib.redirect_stdout(_devnull), \
contextlib.redirect_stderr(_devnull):
# Inherit the parent agent's live runtime (provider, model,
+1 -1
View File
@@ -81,7 +81,7 @@ def build_catalog() -> dict:
def main() -> int:
catalog = build_catalog()
os.makedirs(os.path.dirname(OUTPUT_PATH), exist_ok=True)
with open(OUTPUT_PATH, "w") as fh:
with open(OUTPUT_PATH, "w", encoding="utf-8") as fh:
json.dump(catalog, fh, indent=2)
fh.write("\n")
+1 -1
View File
@@ -304,7 +304,7 @@ def main():
}
os.makedirs(os.path.dirname(OUTPUT_PATH), exist_ok=True)
with open(OUTPUT_PATH, "w") as f:
with open(OUTPUT_PATH, "w", encoding="utf-8") as f:
json.dump(index, f, separators=(",", ":"), ensure_ascii=False)
elapsed = time.time() - overall_start
+1 -1
View File
@@ -291,7 +291,7 @@ def check_release_file(release_file, all_contributors):
missing: set of handles NOT found in the file
"""
try:
content = Path(release_file).read_text()
content = Path(release_file).read_text(encoding="utf-8")
except FileNotFoundError:
print(f" [error] Release file not found: {release_file}", file=sys.stderr)
return set(), set(all_contributors)
+1 -1
View File
@@ -242,7 +242,7 @@ def check_config(groq_key, eleven_key):
if config_path.exists():
try:
import yaml
with open(config_path) as f:
with open(config_path, encoding="utf-8") as f:
cfg = yaml.safe_load(f) or {}
stt_provider = cfg.get("stt", {}).get("provider", "local")
+414 -65
View File
@@ -145,15 +145,30 @@ function Test-Python {
# Python not found — use uv to install it (no admin needed!)
Write-Info "Python $PythonVersion not found, installing via uv..."
try {
# Temporarily relax ErrorActionPreference: uv writes download progress
# to stderr, and with $ErrorActionPreference = "Stop" PowerShell wraps
# those stderr lines as ErrorRecord objects via 2>&1, then throws a
# terminating exception — even when uv exits 0. This caused fresh
# installs to fail on the first run despite Python being installed
# successfully. We verify success with `uv python find` afterwards
# which is the reliable signal regardless of exit code semantics.
$prevEAP = $ErrorActionPreference
$ErrorActionPreference = "Continue"
$uvOutput = & $UvCmd python install $PythonVersion 2>&1
if ($LASTEXITCODE -eq 0) {
$pythonPath = & $UvCmd python find $PythonVersion 2>$null
if ($pythonPath) {
$ver = & $pythonPath --version 2>$null
Write-Success "Python installed: $ver"
return $true
}
} else {
$uvExitCode = $LASTEXITCODE
$ErrorActionPreference = $prevEAP
# Check if Python is now available (more reliable than exit code
# since uv may return non-zero due to "already installed" etc.)
$pythonPath = & $UvCmd python find $PythonVersion 2>$null
if ($pythonPath) {
$ver = & $pythonPath --version 2>$null
Write-Success "Python installed: $ver"
return $true
}
# uv ran but Python still not findable — show what happened
if ($uvExitCode -ne 0) {
Write-Warn "uv python install output:"
Write-Host $uvOutput -ForegroundColor DarkGray
}
@@ -191,19 +206,213 @@ function Test-Python {
return $false
}
function Test-Git {
function Install-Git {
<#
.SYNOPSIS
Ensure Git (and Git Bash) are installed. Git for Windows bundles bash.exe
which Hermes uses to run shell commands.
Priority order (deliberately simple no winget, no registry, no system
package manager):
1. Existing ``git`` on PATH use it as-is (the common fast path).
2. Download **PortableGit** from the official git-for-windows GitHub
release (self-extracting 7z.exe) and unpack it to
``%LOCALAPPDATA%\hermes\git`` never touches system Git, never
requires admin, works even on locked-down machines and machines
with a broken system Git install.
**Why PortableGit, not MinGit:** MinGit is the minimal-automation
distribution and ships ONLY ``git.exe`` no bash, no POSIX utilities.
Hermes needs ``bash.exe`` to run shell commands. PortableGit is the
full Git for Windows distribution without the installer UI; it ships
``git.exe`` + ``bash.exe`` + ``sh``, ``awk``, ``sed``, ``grep``, ``curl``,
``ssh``, etc. in ``usr\bin\``.
We deliberately skip winget because it fails badly when the system Git
install is in a half-installed state (partially registered, or uninstall-
blocked). Owning the Hermes copy of Git ourselves is predictable and
recoverable: if it ever breaks, ``Remove-Item %LOCALAPPDATA%\hermes\git``
and re-running this installer fully recovers.
After install we locate ``bash.exe`` and persist the path in
``HERMES_GIT_BASH_PATH`` (User scope) so Hermes can find it in a fresh
shell without a second PATH refresh.
#>
Write-Info "Checking Git..."
if (Get-Command git -ErrorAction SilentlyContinue) {
$version = git --version
Write-Success "Git found ($version)"
Set-GitBashEnvVar
return $true
}
Write-Err "Git not found"
Write-Info "Please install Git from:"
Write-Info " https://git-scm.com/download/win"
return $false
# Download PortableGit into $HermesHome\git. Always works as long as
# we can reach github.com — no admin, no winget, no reliance on the
# user's possibly-broken system Git install.
Write-Info "Git not found — downloading PortableGit to $HermesHome\git\ ..."
Write-Info "(no admin rights required; isolated from any system Git install)"
try {
$arch = if ([Environment]::Is64BitOperatingSystem) {
# Detect ARM64 vs x64 explicitly; PortableGit ships separate assets.
if ($env:PROCESSOR_ARCHITECTURE -eq "ARM64" -or $env:PROCESSOR_ARCHITEW6432 -eq "ARM64") {
"arm64"
} else {
"64-bit"
}
} else {
# PortableGit does not ship a 32-bit build — fall back to MinGit 32-bit
# with a warning that bash-based features will be unavailable.
"32-bit-mingit"
}
$releaseApi = "https://api.github.com/repos/git-for-windows/git/releases/latest"
$release = Invoke-RestMethod -Uri $releaseApi -UseBasicParsing -Headers @{ "User-Agent" = "hermes-installer" }
if ($arch -eq "32-bit-mingit") {
Write-Warn "32-bit Windows detected — PortableGit is 64-bit only. Installing MinGit 32-bit as a last resort; bash-dependent Hermes features (terminal tool, agent-browser) will not work on this machine."
$assetPattern = "MinGit-*-32-bit.zip"
$downloadIsZip = $true
} elseif ($arch -eq "arm64") {
$assetPattern = "PortableGit-*-arm64.7z.exe"
$downloadIsZip = $false
} else {
$assetPattern = "PortableGit-*-64-bit.7z.exe"
$downloadIsZip = $false
}
$asset = $release.assets | Where-Object { $_.name -like $assetPattern } | Select-Object -First 1
if (-not $asset) {
throw "Could not find $assetPattern in latest git-for-windows release"
}
$downloadUrl = $asset.browser_download_url
$downloadExt = if ($downloadIsZip) { "zip" } else { "7z.exe" }
$tmpFile = "$env:TEMP\$($asset.name)"
$gitDir = "$HermesHome\git"
Write-Info "Downloading $($asset.name) ($([math]::Round($asset.size / 1MB, 1)) MB)..."
Invoke-WebRequest -Uri $downloadUrl -OutFile $tmpFile -UseBasicParsing
if (Test-Path $gitDir) {
Write-Info "Removing previous Git install at $gitDir ..."
Remove-Item -Recurse -Force $gitDir
}
New-Item -ItemType Directory -Path $gitDir -Force | Out-Null
if ($downloadIsZip) {
Expand-Archive -Path $tmpFile -DestinationPath $gitDir -Force
} else {
# PortableGit is a self-extracting 7z archive. Invoke it with
# `-o<target> -y` (silent) to extract to $gitDir. No 7z install
# required; it's fully self-contained.
Write-Info "Extracting PortableGit to $gitDir ..."
$extractProc = Start-Process -FilePath $tmpFile `
-ArgumentList "-o`"$gitDir`"", "-y" `
-NoNewWindow -Wait -PassThru
if ($extractProc.ExitCode -ne 0) {
throw "PortableGit extraction failed (exit code $($extractProc.ExitCode))"
}
}
Remove-Item -Force $tmpFile -ErrorAction SilentlyContinue
# PortableGit layout: cmd\git.exe + bin\bash.exe + usr\bin\ (coreutils)
# MinGit layout: cmd\git.exe + usr\bin\bash.exe (if present)
$gitExe = "$gitDir\cmd\git.exe"
if (-not (Test-Path $gitExe)) {
throw "Git extraction did not produce git.exe at $gitExe"
}
# Add to session PATH so the rest of this install run can use git.
$env:Path = "$gitDir\cmd;$env:Path"
# Persist to User PATH so fresh shells see it. PortableGit needs
# cmd\ (for git.exe), bin\ (for bash.exe + core tools), and
# usr\bin\ (for perl, ssh, curl, and other POSIX coreutils).
$newPathEntries = @(
"$gitDir\cmd",
"$gitDir\bin",
"$gitDir\usr\bin"
)
$userPath = [Environment]::GetEnvironmentVariable("Path", "User")
$userPathItems = if ($userPath) { $userPath -split ";" } else { @() }
$changed = $false
foreach ($entry in $newPathEntries) {
if ($userPathItems -notcontains $entry) {
$userPathItems += $entry
$changed = $true
}
}
if ($changed) {
[Environment]::SetEnvironmentVariable("Path", ($userPathItems -join ";"), "User")
}
$version = & $gitExe --version
Write-Success "Git $version installed to $gitDir (portable, user-scoped)"
Set-GitBashEnvVar
return $true
} catch {
Write-Err "Could not install portable Git: $_"
Write-Info ""
Write-Info "Fallback: install Git manually from https://git-scm.com/download/win"
Write-Info "then re-run this installer. Hermes needs Git Bash on Windows to run"
Write-Info "shell commands (same as Claude Code and other coding agents)."
return $false
}
}
function Set-GitBashEnvVar {
<#
.SYNOPSIS
Locate ``bash.exe`` from an already-installed Git and persist the path in
``HERMES_GIT_BASH_PATH`` (User env scope) so Hermes can find it even before
PATH propagation completes in a newly-spawned shell.
#>
$candidates = @()
# Our own portable Git install is ALWAYS checked first, so a broken
# system Git doesn't hijack us. If the user had a working system Git
# we'd have returned early from Install-Git's fast path and never called
# this with a system-Git-only installation anyway.
#
# Layouts:
# PortableGit (our default): $HermesHome\git\bin\bash.exe
# MinGit (32-bit fallback): $HermesHome\git\usr\bin\bash.exe
$candidates += "$HermesHome\git\bin\bash.exe" # PortableGit layout (primary)
$candidates += "$HermesHome\git\usr\bin\bash.exe" # MinGit / PortableGit usr\bin fallback
# git.exe on PATH can tell us where the install root is
$gitCmd = Get-Command git -ErrorAction SilentlyContinue
if ($gitCmd) {
$gitExe = $gitCmd.Source
# Git for Windows (full installer): <root>\cmd\git.exe + <root>\bin\bash.exe
# MinGit: <root>\cmd\git.exe + <root>\usr\bin\bash.exe
$gitRoot = Split-Path (Split-Path $gitExe -Parent) -Parent
$candidates += "$gitRoot\bin\bash.exe"
$candidates += "$gitRoot\usr\bin\bash.exe"
}
# Standard system install locations as a final fallback. Note:
# ProgramFiles(x86) can't be referenced via ${env:...} string interpolation
# because of the parens — use [Environment]::GetEnvironmentVariable().
$candidates += "${env:ProgramFiles}\Git\bin\bash.exe"
$pf86 = [Environment]::GetEnvironmentVariable("ProgramFiles(x86)")
if ($pf86) { $candidates += "$pf86\Git\bin\bash.exe" }
$candidates += "${env:LocalAppData}\Programs\Git\bin\bash.exe"
foreach ($candidate in $candidates) {
if ($candidate -and (Test-Path $candidate)) {
[Environment]::SetEnvironmentVariable("HERMES_GIT_BASH_PATH", $candidate, "User")
$env:HERMES_GIT_BASH_PATH = $candidate
Write-Info "Set HERMES_GIT_BASH_PATH=$candidate"
return
}
}
Write-Warn "Could not locate bash.exe — Hermes may not find Git Bash."
Write-Info "If needed, set HERMES_GIT_BASH_PATH manually to your bash.exe path."
}
function Test-Node {
@@ -411,21 +620,71 @@ function Install-SystemPackages {
function Install-Repository {
Write-Info "Installing to $InstallDir..."
$didUpdate = $false
if (Test-Path $InstallDir) {
# Test-Path "$InstallDir\.git" returns True when .git is a file OR a
# directory OR a symlink OR a submodule-style gitfile — and also when
# it's a broken stub left over from a failed previous install (e.g.
# a partial Remove-Item that couldn't delete a locked index.lock).
# Validate the repo properly by asking git itself. Two checks
# belt-and-braces: rev-parse AND git status. If either fails the
# repo is broken and we fall through to a fresh clone.
$repoValid = $false
if (Test-Path "$InstallDir\.git") {
Push-Location $InstallDir
try {
# Reset $LASTEXITCODE before the probe so we don't pick up
# a stale 0 from an earlier git call in this session.
$global:LASTEXITCODE = 0
$revParseOut = & git -c windows.appendAtomically=false rev-parse --is-inside-work-tree 2>&1
$revParseOk = ($LASTEXITCODE -eq 0) -and ($revParseOut -match "true")
$global:LASTEXITCODE = 0
$null = & git -c windows.appendAtomically=false status --short 2>&1
$statusOk = ($LASTEXITCODE -eq 0)
if ($revParseOk -and $statusOk) {
$repoValid = $true
}
} catch {}
Pop-Location
}
if ($repoValid) {
Write-Info "Existing installation found, updating..."
Push-Location $InstallDir
git -c windows.appendAtomically=false fetch origin
git -c windows.appendAtomically=false checkout $Branch
git -c windows.appendAtomically=false pull origin $Branch
Pop-Location
try {
git -c windows.appendAtomically=false fetch origin
if ($LASTEXITCODE -ne 0) { throw "git fetch failed (exit $LASTEXITCODE)" }
git -c windows.appendAtomically=false checkout $Branch
if ($LASTEXITCODE -ne 0) { throw "git checkout $Branch failed (exit $LASTEXITCODE)" }
git -c windows.appendAtomically=false pull origin $Branch
if ($LASTEXITCODE -ne 0) { throw "git pull failed (exit $LASTEXITCODE)" }
} finally {
Pop-Location
}
$didUpdate = $true
} else {
Write-Err "Directory exists but is not a git repository: $InstallDir"
Write-Info "Remove it or choose a different directory with -InstallDir"
throw "Directory exists but is not a git repository: $InstallDir"
# Directory exists but isn't a usable git repo. Wipe it and
# fall through to a fresh clone. A leftover ``.git`` stub from
# a partial uninstall used to lock the installer into the
# "update" branch forever, emitting three ``fatal: not a git
# repository`` errors and failing with "not in a git directory".
Write-Warn "Existing directory at $InstallDir is not a valid git repo — replacing it."
try {
Remove-Item -Recurse -Force $InstallDir -ErrorAction Stop
} catch {
Write-Err "Could not remove $InstallDir : $_"
Write-Info "Close any programs that might be using files in $InstallDir (editors,"
Write-Info "terminals, running hermes processes) and try again."
throw
}
}
} else {
}
if (-not $didUpdate) {
$cloneSuccess = $false
# Fix Windows git "copy-fd: write returned: Invalid argument" error.
@@ -446,7 +705,7 @@ function Install-Repository {
if ($LASTEXITCODE -eq 0) { $cloneSuccess = $true }
} catch { }
$env:GIT_SSH_COMMAND = $null
if (-not $cloneSuccess) {
if (Test-Path $InstallDir) { Remove-Item -Recurse -Force $InstallDir -ErrorAction SilentlyContinue }
Write-Info "SSH failed, trying HTTPS..."
@@ -464,18 +723,18 @@ function Install-Repository {
$zipUrl = "https://github.com/NousResearch/hermes-agent/archive/refs/heads/$Branch.zip"
$zipPath = "$env:TEMP\hermes-agent-$Branch.zip"
$extractPath = "$env:TEMP\hermes-agent-extract"
Invoke-WebRequest -Uri $zipUrl -OutFile $zipPath -UseBasicParsing
if (Test-Path $extractPath) { Remove-Item -Recurse -Force $extractPath }
Expand-Archive -Path $zipPath -DestinationPath $extractPath -Force
# GitHub ZIPs extract to repo-branch/ subdirectory
$extractedDir = Get-ChildItem $extractPath -Directory | Select-Object -First 1
if ($extractedDir) {
New-Item -ItemType Directory -Force -Path (Split-Path $InstallDir) -ErrorAction SilentlyContinue | Out-Null
Move-Item $extractedDir.FullName $InstallDir -Force
Write-Success "Downloaded and extracted"
# Initialize git repo so updates work later
Push-Location $InstallDir
git -c windows.appendAtomically=false init 2>$null
@@ -483,10 +742,10 @@ function Install-Repository {
git remote add origin $RepoUrlHttps 2>$null
Pop-Location
Write-Success "Git repo initialized for future updates"
$cloneSuccess = $true
}
# Cleanup temp files
Remove-Item -Force $zipPath -ErrorAction SilentlyContinue
Remove-Item -Recurse -Force $extractPath -ErrorAction SilentlyContinue
@@ -499,7 +758,7 @@ function Install-Repository {
throw "Failed to download repository (tried git clone SSH, HTTPS, and ZIP)"
}
}
# Set per-repo config (harmless if it fails)
Push-Location $InstallDir
git -c windows.appendAtomically=false config windows.appendAtomically false 2>$null
@@ -513,7 +772,7 @@ function Install-Repository {
Write-Success "Submodules ready"
}
Pop-Location
Write-Success "Repository ready"
}
@@ -659,13 +918,21 @@ function Copy-ConfigTemplates {
Write-Info "~/.hermes/config.yaml already exists, keeping it"
}
# Create SOUL.md if it doesn't exist (global persona file)
# Create SOUL.md if it doesn't exist (global persona file).
# IMPORTANT: write without a BOM. Windows PowerShell 5.1's
# ``Set-Content -Encoding UTF8`` writes UTF-8 WITH a byte-order-mark
# (the default PS5 behaviour), and Hermes's prompt-injection scanner
# flags the BOM as an invisible unicode character and refuses to
# load the file. PS7's ``-Encoding utf8NoBOM`` fixes that but we
# don't control which PowerShell version the user has. Go direct
# to .NET with an explicit UTF8Encoding($false) — BOM-free on every
# PowerShell version.
$soulPath = "$HermesHome\SOUL.md"
if (-not (Test-Path $soulPath)) {
@"
$soulContent = @"
# Hermes Agent Persona
<!--
<!--
This file defines the agent's personality and tone.
The agent will embody whatever you write here.
Edit this to customize how Hermes communicates with you.
@@ -678,7 +945,9 @@ Examples:
This file is loaded fresh each message -- no restart needed.
Delete the contents (or this file) to use the default personality.
-->
"@ | Set-Content -Path $soulPath -Encoding UTF8
"@
$utf8NoBom = New-Object System.Text.UTF8Encoding($false)
[System.IO.File]::WriteAllText($soulPath, $soulContent, $utf8NoBom)
Write-Success "Created ~/.hermes/SOUL.md (edit to customize personality)"
}
@@ -708,36 +977,94 @@ function Install-NodeDeps {
Write-Info "Skipping Node.js dependencies (Node not installed)"
return
}
Push-Location $InstallDir
if (Test-Path "package.json") {
Write-Info "Installing Node.js dependencies (browser tools)..."
try {
npm install --silent 2>&1 | Out-Null
Write-Success "Node.js dependencies installed"
} catch {
Write-Warn "npm install failed (browser tools may not work)"
# Resolve npm explicitly to npm.cmd, NOT npm.ps1. Node.js on Windows
# ships BOTH npm.cmd (a batch shim) and npm.ps1 (a PowerShell shim).
# Get-Command's default ordering picks whichever comes first in PATHEXT,
# and on many systems that's .ps1 — but .ps1 requires scripts to be
# enabled in PowerShell's execution policy, which most Windows users
# don't have (the Restricted / RemoteSigned default blocks unsigned
# .ps1 files). .cmd has no such restriction and works on every box.
#
# Strategy: look next to the npm shim we found and prefer npm.cmd if
# it exists in the same directory. Fall back to whatever Get-Command
# returned if we can't find a .cmd sibling.
$npmCmd = Get-Command npm -ErrorAction SilentlyContinue
if (-not $npmCmd) {
Write-Warn "npm not found on PATH — skipping Node.js dependencies."
Write-Info "Open a new PowerShell window and re-run 'hermes setup tools' later."
return
}
$npmExe = $npmCmd.Source
if ($npmExe -like "*.ps1") {
$npmCmdSibling = Join-Path (Split-Path $npmExe -Parent) "npm.cmd"
if (Test-Path $npmCmdSibling) {
Write-Info "Using npm.cmd (PowerShell execution policy blocks npm.ps1)"
$npmExe = $npmCmdSibling
} else {
Write-Warn "Only npm.ps1 available — install may fail if script execution is disabled."
Write-Info " If it fails, either enable PS script execution or install Node via winget."
}
}
# Install TUI dependencies
# Helper: run "npm install" in a given directory and surface the real
# error when it fails. Returns $true on success.
#
# Implementation note: ``Start-Process -FilePath npm.cmd`` fails with
# ``%1 is not a valid Win32 application`` on some PowerShell versions
# because Start-Process bypasses cmd.exe / PATHEXT and expects a real
# PE file. The invocation-operator ``& $npmExe`` routes through the
# PowerShell command pipeline which DOES honour .cmd batch shims, so
# it works uniformly for npm.cmd, npx.cmd, and bare .exe files.
function _Run-NpmInstall([string]$label, [string]$installDir, [string]$logPath, [string]$npmPath) {
Push-Location $installDir
try {
# Redirect ALL output streams to the log file via 2>&1 and then
# ``Tee-Object`` / ``Out-File``. Simpler approach: call npm
# with output redirected and inspect $LASTEXITCODE afterwards.
& $npmPath install --silent *> $logPath
$code = $LASTEXITCODE
if ($code -eq 0) {
Write-Success "$label dependencies installed"
Remove-Item -Force $logPath -ErrorAction SilentlyContinue
return $true
}
Write-Warn "$label npm install failed — exit code $code"
if (Test-Path $logPath) {
$errText = (Get-Content $logPath -Raw -ErrorAction SilentlyContinue)
if ($errText) {
$snippet = if ($errText.Length -gt 1200) { $errText.Substring(0, 1200) + "..." } else { $errText }
Write-Info " npm output:"
foreach ($line in $snippet -split "`n") {
Write-Host " $line" -ForegroundColor DarkGray
}
Write-Info " Full log: $logPath"
}
}
Write-Info "Run manually later: cd `"$installDir`"; npm install"
return $false
} catch {
Write-Warn "$label npm install could not be launched: $_"
return $false
} finally {
Pop-Location
}
}
# Browser tools
if (Test-Path "$InstallDir\package.json") {
Write-Info "Installing Node.js dependencies (browser tools)..."
$browserLog = "$env:TEMP\hermes-npm-browser-$(Get-Random).log"
[void](_Run-NpmInstall "Browser tools" $InstallDir $browserLog $npmExe)
}
# TUI
$tuiDir = "$InstallDir\ui-tui"
if (Test-Path "$tuiDir\package.json") {
Write-Info "Installing TUI dependencies..."
Push-Location $tuiDir
try {
npm install --silent 2>&1 | Out-Null
Write-Success "TUI dependencies installed"
} catch {
Write-Warn "TUI npm install failed (hermes --tui may not work)"
}
Pop-Location
$tuiLog = "$env:TEMP\hermes-npm-tui-$(Get-Random).log"
[void](_Run-NpmInstall "TUI" $tuiDir $tuiLog $npmExe)
}
Pop-Location
}
function Invoke-SetupWizard {
@@ -886,13 +1213,35 @@ function Write-Completion {
function Main {
Write-Banner
# Windows refuses to delete a directory any shell is currently cd'd
# inside — and silently leaves orphan files behind, which then wedge
# "is this a valid git repo" probes on re-install. If the current
# working dir is under $InstallDir, step out to the user's home
# BEFORE doing anything else. Harmless when the user ran the
# installer from somewhere else.
try {
$currentResolved = (Get-Location).ProviderPath
$installResolved = $null
if (Test-Path $InstallDir) {
$installResolved = (Resolve-Path $InstallDir -ErrorAction SilentlyContinue).ProviderPath
}
if ($installResolved -and $currentResolved.ToLower().StartsWith($installResolved.ToLower())) {
Write-Info "Stepping out of $InstallDir so Windows can replace files there if needed..."
Set-Location $env:USERPROFILE
}
} catch {}
if (-not (Install-Uv)) { throw "uv installation failed — cannot continue" }
if (-not (Test-Python)) { throw "Python $PythonVersion not available — cannot continue" }
if (-not (Test-Git)) { throw "Git not found — install from https://git-scm.com/download/win" }
Test-Node # Auto-installs if missing
if (-not (Install-Git)) { throw "Git not available and auto-install failed — install from https://git-scm.com/download/win then re-run" }
# Test-Node always returns $true (sets $script:HasNode on success, emits a
# warning on failure and continues so non-browser installs still work).
# Cast to [void] so the bare return value doesn't print "True" to the
# console between the "Node found" line and the next installer step.
[void](Test-Node)
Install-SystemPackages # ripgrep + ffmpeg in one step
Install-Repository
Install-Venv
Install-Dependencies
@@ -901,7 +1250,7 @@ function Main {
Copy-ConfigTemplates
Invoke-SetupWizard
Start-GatewayIfConfigured
Write-Completion
}
+2 -2
View File
@@ -111,7 +111,7 @@ def summarize(log: Path, since_ts_ms: int) -> dict[str, Any]:
frame_events: list[dict[str, Any]] = []
if not log.exists():
return {"error": f"no log at {log}", "react": [], "frame": []}
for line in log.read_text().splitlines():
for line in log.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line:
continue
@@ -505,7 +505,7 @@ def main() -> int:
if args.save:
path = Path(f"/tmp/perf-{args.save}.json")
path.write_text(json.dumps(metrics, indent=2))
path.write_text(json.dumps(metrics, indent=2), encoding="utf-8")
print(f"\n• saved: {path}")
if args.compare:
+1 -1
View File
@@ -1359,7 +1359,7 @@ def main():
)
if args.output:
Path(args.output).write_text(changelog)
Path(args.output).write_text(changelog, encoding="utf-8")
print(f"Changelog written to {args.output}")
else:
print(changelog)
+132 -1
View File
@@ -152,4 +152,135 @@ class TestRelaunch:
with pytest.raises(SystemExit):
relaunch_mod.relaunch(["--resume", "abc"])
assert calls == [("/usr/bin/hermes", ["/usr/bin/hermes", "--resume", "abc"])]
assert calls == [("/usr/bin/hermes", ["/usr/bin/hermes", "--resume", "abc"])]
def test_windows_uses_subprocess_not_execvp(self, monkeypatch):
"""On Windows, os.execvp raises OSError "Exec format error" when the
target is a .cmd shim or console-script wrapper (both common for
hermes). relaunch() must detect win32 and use subprocess.run +
sys.exit instead."""
monkeypatch.setattr(relaunch_mod.sys, "platform", "win32")
monkeypatch.setattr(relaunch_mod, "resolve_hermes_bin", lambda: r"C:\Users\test\hermes.exe")
import subprocess as _subprocess
captured_argv = []
def fake_subprocess_run(argv, **kwargs):
captured_argv.append(list(argv))
class _Result:
returncode = 0
return _Result()
monkeypatch.setattr(_subprocess, "run", fake_subprocess_run)
# execvp MUST NOT be called on Windows — route must go through subprocess
execvp_calls = []
def fake_execvp(*args, **kwargs):
execvp_calls.append(args)
raise AssertionError("os.execvp must not be called on Windows")
monkeypatch.setattr(relaunch_mod.os, "execvp", fake_execvp)
with pytest.raises(SystemExit) as exc_info:
relaunch_mod.relaunch(["chat"])
assert exc_info.value.code == 0
assert execvp_calls == []
assert captured_argv == [[r"C:\Users\test\hermes.exe", "chat"]]
def test_windows_propagates_child_exit_code(self, monkeypatch):
"""A non-zero exit from the child should flow through to sys.exit."""
monkeypatch.setattr(relaunch_mod.sys, "platform", "win32")
monkeypatch.setattr(relaunch_mod, "resolve_hermes_bin", lambda: r"C:\hermes.exe")
import subprocess as _subprocess
def fake_run(argv, **kwargs):
class _Result:
returncode = 42
return _Result()
monkeypatch.setattr(_subprocess, "run", fake_run)
monkeypatch.setattr(relaunch_mod.os, "execvp", lambda *a, **kw: None)
with pytest.raises(SystemExit) as exc_info:
relaunch_mod.relaunch(["chat"])
assert exc_info.value.code == 42
def test_windows_surfaces_oserror_with_help(self, monkeypatch, capsys):
"""When subprocess itself raises OSError (file-not-found / bad format),
we must NOT let it bubble up as a cryptic traceback print a
user-readable hint and sys.exit(1)."""
monkeypatch.setattr(relaunch_mod.sys, "platform", "win32")
monkeypatch.setattr(relaunch_mod, "resolve_hermes_bin", lambda: r"C:\missing.exe")
import subprocess as _subprocess
def fake_run(argv, **kwargs):
raise OSError(2, "No such file or directory")
monkeypatch.setattr(_subprocess, "run", fake_run)
monkeypatch.setattr(relaunch_mod.os, "execvp", lambda *a, **kw: None)
with pytest.raises(SystemExit) as exc_info:
relaunch_mod.relaunch(["chat"])
assert exc_info.value.code == 1
err = capsys.readouterr().err
assert "relaunch failed" in err
assert "open a new terminal" in err.lower() or "path" in err.lower()
class TestResolveHermesBinWindowsPyGuard:
"""On Windows, resolve_hermes_bin MUST NOT return a .py path.
os.access(x, os.X_OK) returns True for .py files on Windows because
PATHEXT includes .py when the Python launcher is installed but
subprocess.run can't actually exec a .py directly, so the relaunch
would fail with the cryptic "%1 is not a valid Win32 application" error.
"""
def test_windows_rejects_py_argv0_falls_through_to_path(self, monkeypatch, tmp_path):
"""On Windows, if sys.argv[0] is a .py file, we must skip the
argv[0] fast-path and fall through to PATH / python -m."""
# Build a fake .py script that "passes" the isfile + X_OK checks.
script = tmp_path / "main.py"
script.write_text("# stub")
monkeypatch.setattr(relaunch_mod.sys, "platform", "win32")
monkeypatch.setattr(relaunch_mod.sys, "argv", [str(script), "chat"])
# Force PATH lookup to return a hermes.exe so the test doesn't
# exercise the None-fallback path (that's a separate test).
monkeypatch.setattr(
relaunch_mod.shutil, "which",
lambda name: r"C:\venv\Scripts\hermes.exe" if name == "hermes" else None,
)
bin_path = relaunch_mod.resolve_hermes_bin()
# Must NOT be the .py — must be the hermes.exe PATH entry.
assert bin_path == r"C:\venv\Scripts\hermes.exe"
def test_posix_still_accepts_py_argv0(self, monkeypatch, tmp_path):
"""POSIX behaviour unchanged: argv[0] pointing at an executable
script (including .py with a shebang + chmod +x) is fine to return
because POSIX exec can route through the shebang line."""
if sys.platform == "win32":
pytest.skip("POSIX semantics")
script = tmp_path / "hermes"
script.write_text("#!/usr/bin/env python3\n")
script.chmod(0o755)
monkeypatch.setattr(relaunch_mod.sys, "argv", [str(script), "chat"])
assert relaunch_mod.resolve_hermes_bin() == str(script)
def test_windows_py_argv0_with_no_hermes_on_path_returns_none(self, monkeypatch, tmp_path):
"""Bulletproof fallback: if argv0 is .py on Windows AND hermes.exe
isn't on PATH, return None so the caller falls back to
python -m hermes_cli.main."""
script = tmp_path / "main.py"
script.write_text("# stub")
monkeypatch.setattr(relaunch_mod.sys, "platform", "win32")
monkeypatch.setattr(relaunch_mod.sys, "argv", [str(script), "chat"])
monkeypatch.setattr(relaunch_mod.shutil, "which", lambda name: None)
assert relaunch_mod.resolve_hermes_bin() is None
+297
View File
@@ -0,0 +1,297 @@
"""Tests for hermes_bootstrap — Windows UTF-8 stdio shim.
The bootstrap module is imported at the top of every Hermes entry point
(hermes, hermes-agent, hermes-acp, gateway, batch_runner, cli.py). It
fixes Python's Windows UTF-8 defaults so print("café") doesn't crash and
subprocess children inherit UTF-8 mode.
Key invariants covered by these tests:
1. Windows: env vars get set, stdio reconfigured, non-ASCII print works
2. POSIX: complete no-op (we don't touch LANG/LC_* or anything else)
3. Idempotent: safe to call multiple times
4. Respects user opt-out: if the user explicitly sets PYTHONUTF8=0 or
PYTHONIOENCODING=something-else, we leave those alone
5. Load order: every Hermes entry point imports hermes_bootstrap as its
first non-docstring import (before anything that might do file I/O
or print to stdout)
"""
from __future__ import annotations
import io
import os
import subprocess
import sys
import textwrap
import unittest.mock as mock
import pytest
# Import the module under test via an import-time side-effect check path.
# We need to be able to reset its state between tests, so we import it
# fresh in each test that manipulates _IS_WINDOWS.
def _fresh_import():
"""Return a freshly-imported hermes_bootstrap module.
Drops any cached copy from sys.modules first so module-level code
runs again and the platform check re-evaluates.
"""
sys.modules.pop("hermes_bootstrap", None)
import hermes_bootstrap # noqa: WPS433
return hermes_bootstrap
class TestWindowsBehavior:
"""Windows: the bootstrap does its job."""
@pytest.mark.skipif(
sys.platform != "win32",
reason="Windows-specific behavior",
)
def test_env_vars_set_on_windows(self, monkeypatch):
# Clear any pre-existing values and re-run bootstrap.
monkeypatch.delenv("PYTHONUTF8", raising=False)
monkeypatch.delenv("PYTHONIOENCODING", raising=False)
hb = _fresh_import()
# Module-level apply_windows_utf8_bootstrap() ran during import.
assert os.environ.get("PYTHONUTF8") == "1"
assert os.environ.get("PYTHONIOENCODING") == "utf-8"
assert hb._bootstrap_applied is True
@pytest.mark.skipif(
sys.platform != "win32",
reason="Windows-specific behavior",
)
def test_stdout_reconfigured_to_utf8_on_windows(self):
# The live process's stdout should now be UTF-8 (the Hermes CLI
# runs on Windows with a pytest console that's cp1252 by default).
# If reconfigure succeeded, sys.stdout.encoding is 'utf-8'.
_fresh_import()
# pytest may capture stdout, which makes encoding check flaky —
# so instead verify the reconfigure call succeeded on the real
# stream by attempting the failure case.
out = sys.stdout
reconfigure = getattr(out, "reconfigure", None)
if reconfigure is None:
pytest.skip("pytest replaced sys.stdout with a non-reconfigurable stream")
# After bootstrap, encoding should be utf-8 (or the reconfigure
# skipped because pytest's capture already set it to utf-8).
assert out.encoding.lower() in {"utf-8", "utf8"}, (
f"stdout encoding is {out.encoding!r} — bootstrap should have "
"reconfigured it to UTF-8"
)
@pytest.mark.skipif(
sys.platform != "win32",
reason="Windows-specific behavior",
)
def test_child_process_inherits_utf8_mode(self):
"""A subprocess spawned from this process should inherit
PYTHONUTF8=1 and be able to print non-ASCII to stdout."""
_fresh_import()
# Non-ASCII chars that would crash under cp1252: arrow, emoji.
script = textwrap.dedent("""
import sys
print("em-dash \\u2014 arrow \\u2192 emoji \\U0001f680")
sys.exit(0)
""").strip()
# Don't pass env= — let the child inherit os.environ, which
# now contains PYTHONUTF8=1 courtesy of the bootstrap.
result = subprocess.run(
[sys.executable, "-c", script],
capture_output=True,
timeout=15,
)
assert result.returncode == 0, (
f"Child crashed printing non-ASCII despite UTF-8 bootstrap:\n"
f" stdout: {result.stdout!r}\n"
f" stderr: {result.stderr!r}"
)
decoded = result.stdout.decode("utf-8")
assert "\u2014" in decoded
assert "\u2192" in decoded
assert "\U0001f680" in decoded
class TestUserOptOut:
"""If the user has explicitly set PYTHONUTF8 / PYTHONIOENCODING in
their environment, we respect that (setdefault, not overwrite)."""
@pytest.mark.skipif(
sys.platform != "win32",
reason="Only meaningful on Windows where we'd otherwise set these",
)
def test_user_pythonutf8_zero_preserved(self, monkeypatch):
monkeypatch.setenv("PYTHONUTF8", "0")
_fresh_import()
assert os.environ["PYTHONUTF8"] == "0", (
"bootstrap must not overwrite an explicit user setting"
)
@pytest.mark.skipif(
sys.platform != "win32",
reason="Only meaningful on Windows where we'd otherwise set these",
)
def test_user_pythonioencoding_preserved(self, monkeypatch):
monkeypatch.setenv("PYTHONIOENCODING", "latin-1")
_fresh_import()
assert os.environ["PYTHONIOENCODING"] == "latin-1"
class TestPosixNoOp:
"""POSIX: zero behavior change. We don't touch LANG, LC_*, or any
stdio. The goal is that Linux/macOS behave identically before and
after this module is imported."""
def test_noop_on_fake_posix(self, monkeypatch):
"""Even when imported, the bootstrap function must return False
and leave env untouched when _IS_WINDOWS is False."""
hb = _fresh_import()
# Reset + fake POSIX
hb._IS_WINDOWS = False
hb._bootstrap_applied = False
monkeypatch.delenv("PYTHONUTF8", raising=False)
monkeypatch.delenv("PYTHONIOENCODING", raising=False)
result = hb.apply_windows_utf8_bootstrap()
assert result is False
assert "PYTHONUTF8" not in os.environ
assert "PYTHONIOENCODING" not in os.environ
assert hb._bootstrap_applied is False
@pytest.mark.skipif(
sys.platform == "win32",
reason="Real POSIX required for this check",
)
def test_real_posix_bootstrap_is_noop(self, monkeypatch):
"""On actual Linux/macOS, importing the module must not set
PYTHONUTF8 or reconfigure stdio."""
monkeypatch.delenv("PYTHONUTF8", raising=False)
monkeypatch.delenv("PYTHONIOENCODING", raising=False)
hb = _fresh_import()
assert hb._bootstrap_applied is False
assert "PYTHONUTF8" not in os.environ
assert "PYTHONIOENCODING" not in os.environ
class TestIdempotence:
"""Calling apply_windows_utf8_bootstrap() multiple times must be safe."""
def test_second_call_returns_false(self):
hb = _fresh_import()
# First call already happened at import time.
result = hb.apply_windows_utf8_bootstrap()
assert result is False, (
"Second call should return False (idempotent no-op)"
)
def test_no_exceptions_on_repeated_calls(self):
hb = _fresh_import()
for _ in range(5):
hb.apply_windows_utf8_bootstrap()
class TestStdioReconfigureErrorHandling:
"""If sys.stdout/stderr/stdin have been replaced with streams that
don't support reconfigure (e.g. by a test harness), the bootstrap
must degrade gracefully rather than crash."""
def test_non_reconfigurable_stream_does_not_crash(self, monkeypatch):
"""Replace sys.stdout with a BytesIO (no reconfigure method),
then run the bootstrap and make sure it doesn't raise."""
hb = _fresh_import()
hb._IS_WINDOWS = True
hb._bootstrap_applied = False
fake = io.BytesIO() # no .reconfigure attribute
monkeypatch.setattr(sys, "stdout", fake)
try:
# Must not raise.
hb.apply_windows_utf8_bootstrap()
except Exception as exc:
pytest.fail(f"bootstrap raised on non-reconfigurable stdout: {exc}")
def test_reconfigure_oserror_is_caught(self, monkeypatch):
"""If reconfigure() itself raises (closed stream, etc.), swallow
the error the env-var half of the fix still applies."""
hb = _fresh_import()
hb._IS_WINDOWS = True
hb._bootstrap_applied = False
class _BrokenStream:
encoding = "utf-8"
def reconfigure(self, **kwargs):
raise OSError("simulated: stream already closed")
monkeypatch.setattr(sys, "stdout", _BrokenStream())
monkeypatch.setattr(sys, "stderr", _BrokenStream())
# Must not raise.
hb.apply_windows_utf8_bootstrap()
class TestEntryPointsImportBootstrap:
"""Every Hermes entry point must import hermes_bootstrap as its
first non-docstring import. We check this by scanning source files
rather than invoking the entry points (which would require a full
agent context)."""
# Entry points that invoke Hermes as a process. Each one must
# import hermes_bootstrap before doing any file I/O or stdout writes.
ENTRY_POINTS = [
"hermes_cli/main.py", # hermes CLI (console_script)
"run_agent.py", # hermes-agent (console_script)
"acp_adapter/entry.py", # hermes-acp (console_script)
"gateway/run.py", # gateway
"batch_runner.py", # batch mode
"cli.py", # legacy direct-launch CLI
]
@pytest.mark.parametrize("path", ENTRY_POINTS)
def test_entry_point_imports_bootstrap(self, path):
"""The file must contain 'import hermes_bootstrap' and that
line must appear before the first 'import' of anything else.
We're lenient about the docstring (can be arbitrarily long) and
about comment lines just need to verify the first import
statement is the bootstrap.
"""
# Resolve relative to the hermes-agent repo root. Tests live
# at tests/test_hermes_bootstrap.py, so go up one dir.
import pathlib
here = pathlib.Path(__file__).resolve()
repo_root = here.parent.parent # tests/ -> repo root
full_path = repo_root / path
assert full_path.exists(), f"entry point missing: {full_path}"
source = full_path.read_text(encoding="utf-8")
# Find the first non-comment, non-blank line that starts with
# 'import ' or 'from '. It must be 'import hermes_bootstrap'.
import tokenize
import ast
tree = ast.parse(source)
first_import_node = None
for node in ast.iter_child_nodes(tree):
if isinstance(node, (ast.Import, ast.ImportFrom)):
first_import_node = node
break
assert first_import_node is not None, (
f"{path}: no top-level imports found at all"
)
if isinstance(first_import_node, ast.Import):
first_import_name = first_import_node.names[0].name
else: # ImportFrom
first_import_name = first_import_node.module or ""
assert first_import_name == "hermes_bootstrap", (
f"{path}: first top-level import is {first_import_name!r}, "
f"but it must be 'hermes_bootstrap' so UTF-8 stdio is "
f"configured before anything else initializes. Move the "
f"'import hermes_bootstrap' line to be the first import."
)
+115
View File
@@ -0,0 +1,115 @@
"""Tests for ruff lint config — guards against accidental rule removal.
PLW1514 (unspecified-encoding) was enabled after a debug session on
Windows turned up three separate UTF-8 regressions in execute_code.
The rule catches bare ``open()`` / ``read_text()`` / ``write_text()``
calls that default to locale encoding cp1252 on Windows which
silently corrupts non-ASCII content.
These tests ensure:
1. PLW1514 stays in ``[tool.ruff.lint.select]``
2. The CI workflow's blocking step still invokes ``ruff check .``
3. pyproject.toml has ``preview = true`` (required PLW1514 is a
preview rule in ruff 0.15.x)
If someone removes any of these, CI stops enforcing UTF-8-explicit
opens and we're back to the original Windows-regression trap.
"""
from __future__ import annotations
import pathlib
import pytest
try:
import tomllib # Python 3.11+
except ImportError: # pragma: no cover — 3.10 and earlier
import tomli as tomllib # type: ignore
REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent
def _load_pyproject() -> dict:
with open(REPO_ROOT / "pyproject.toml", "rb") as fh:
return tomllib.load(fh)
class TestRuffConfig:
def test_plw1514_is_in_select_list(self):
"""pyproject.toml must keep PLW1514 in [tool.ruff.lint.select]."""
cfg = _load_pyproject()
selected = (
cfg.get("tool", {})
.get("ruff", {})
.get("lint", {})
.get("select", [])
)
assert "PLW1514" in selected, (
"PLW1514 (unspecified-encoding) was removed from "
"[tool.ruff.lint.select]. This rule blocks bare open() calls "
"that default to locale encoding on Windows — removing it "
"re-opens a class of UTF-8 bugs we already paid to close. "
"If you genuinely want to remove it, delete this test in the "
"same commit so the intent is deliberate."
)
def test_preview_mode_enabled(self):
"""PLW1514 is a preview rule in ruff 0.15.x — preview=true is
required for it to actually run."""
cfg = _load_pyproject()
ruff_cfg = cfg.get("tool", {}).get("ruff", {})
assert ruff_cfg.get("preview") is True, (
"[tool.ruff] preview=true is required — PLW1514 is a preview "
"rule and silently becomes a no-op without it. If this ever "
"becomes a stable rule, you can drop preview=true but must "
"verify PLW1514 still fires in a sample test run first."
)
class TestLintWorkflow:
WORKFLOW_PATH = REPO_ROOT / ".github" / "workflows" / "lint.yml"
def test_workflow_exists(self):
assert self.WORKFLOW_PATH.exists(), (
f"CI workflow missing: {self.WORKFLOW_PATH}"
)
def test_workflow_has_blocking_ruff_step(self):
"""The workflow must run a blocking ``ruff check .`` step
(one without --exit-zero) so violations fail the job."""
content = self.WORKFLOW_PATH.read_text(encoding="utf-8")
# Look for the blocking step's named line + its command. We want
# at least one ``ruff check .`` that does NOT have ``--exit-zero``
# nearby.
import re
# Split into lines and find ruff check invocations
lines = content.splitlines()
found_blocking = False
for i, line in enumerate(lines):
stripped = line.strip()
if stripped.startswith("ruff check") and "--exit-zero" not in stripped:
# Also check it's not piped to `|| true` which would mask
# the exit code.
window = " ".join(lines[i:i + 3])
if "|| true" not in window:
found_blocking = True
break
assert found_blocking, (
"lint.yml no longer contains a blocking ``ruff check .`` step "
"(one without --exit-zero and not masked by || true). "
"Restore it — the PLW1514 rule is only useful if CI actually "
"fails on violation."
)
def test_workflow_yaml_is_valid(self):
"""Workflow file must parse as valid YAML (can't ship a broken
CI config to main)."""
import yaml
content = self.WORKFLOW_PATH.read_text(encoding="utf-8")
try:
parsed = yaml.safe_load(content)
except yaml.YAMLError as exc:
pytest.fail(f"lint.yml is not valid YAML: {exc}")
assert isinstance(parsed, dict)
assert "jobs" in parsed
+9 -1
View File
@@ -340,7 +340,15 @@ class TestRunBrowserCommandPathConstruction:
_run_browser_command("test-task", "navigate", ["https://example.com"])
assert captured_cmd is not None
assert captured_cmd[:2] == ["npx", "agent-browser"]
# The prefix must split "npx agent-browser" into two argv items.
# On POSIX shutil.which("npx") returns the absolute path if npx is on
# PATH (which the test's patched PATH always contains when the system
# has it installed). The important invariant is that the second
# argv item is the package name "agent-browser", not a merged
# "npx agent-browser" string — that's what Popen needs.
assert len(captured_cmd) >= 2
assert captured_cmd[0].endswith("npx") or captured_cmd[0] == "npx"
assert captured_cmd[1] == "agent-browser"
assert captured_cmd[2:6] == [
"--session",
"test-session",
+8 -2
View File
@@ -774,11 +774,17 @@ class TestEnvVarFiltering(unittest.TestCase):
class TestExecuteCodeEdgeCases(unittest.TestCase):
def test_windows_returns_error(self):
"""On Windows (or when SANDBOX_AVAILABLE is False), returns error JSON."""
"""When SANDBOX_AVAILABLE is False (e.g. when the backend deems
the sandbox unusable for this environment), execute_code returns
an error JSON with a readable message pointing the caller at
regular tool calls. Previously this was a Windows-only gate;
execute_code now works on Windows via loopback TCP, so the
error is only emitted when SANDBOX_AVAILABLE is explicitly
flipped off (e.g. for future platform-specific disables)."""
with patch("tools.code_execution_tool.SANDBOX_AVAILABLE", False):
result = json.loads(execute_code("print('hi')", task_id="test"))
self.assertIn("error", result)
self.assertIn("Windows", result["error"])
self.assertIn("unavailable", result["error"].lower())
def test_whitespace_only_code(self):
result = json.loads(execute_code(" \n\t ", task_id="test"))
+30 -2
View File
@@ -131,6 +131,12 @@ class TestResolveChildPython(unittest.TestCase):
def test_project_with_virtualenv_picks_venv_python(self):
"""Project mode + VIRTUAL_ENV pointing at a real venv → that python."""
if sys.platform == "win32":
pytest.skip(
"Creates symlinks and assumes POSIX venv layout (bin/python). "
"Windows venvs use Scripts/python.exe and symlink creation "
"requires elevated privileges (WinError 1314)."
)
import tempfile, pathlib
with tempfile.TemporaryDirectory() as td:
fake_venv = pathlib.Path(td)
@@ -154,6 +160,12 @@ class TestResolveChildPython(unittest.TestCase):
def test_project_prefers_virtualenv_over_conda(self):
"""If both VIRTUAL_ENV and CONDA_PREFIX are set, VIRTUAL_ENV wins."""
if sys.platform == "win32":
pytest.skip(
"Creates symlinks and assumes POSIX venv layout (bin/python). "
"Windows venvs use Scripts/python.exe and symlink creation "
"requires elevated privileges (WinError 1314)."
)
import tempfile, pathlib
with tempfile.TemporaryDirectory() as ve_td, tempfile.TemporaryDirectory() as conda_td:
ve = pathlib.Path(ve_td)
@@ -257,7 +269,15 @@ class TestModeAwareSchema(unittest.TestCase):
# Integration: what actually happens when execute_code runs per mode
# ---------------------------------------------------------------------------
@pytest.mark.skipif(sys.platform == "win32", reason="execute_code is POSIX-only")
@pytest.mark.skipif(
sys.platform == "win32",
reason=(
"Assumes POSIX venv layout (bin/python) and symlink creation "
"privileges. execute_code itself works on Windows — these "
"integration tests just haven't been ported to the Scripts/"
"python.exe layout yet."
),
)
class TestExecuteCodeModeIntegration(unittest.TestCase):
"""End-to-end: verify the subprocess actually runs where we expect."""
@@ -351,7 +371,15 @@ class TestExecuteCodeModeIntegration(unittest.TestCase):
# changes CWD + interpreter, not the security posture.
# ---------------------------------------------------------------------------
@pytest.mark.skipif(sys.platform == "win32", reason="execute_code is POSIX-only")
@pytest.mark.skipif(
sys.platform == "win32",
reason=(
"Assumes POSIX venv layout (bin/python) and symlink creation "
"privileges. execute_code itself works on Windows — these "
"integration tests just haven't been ported to the Scripts/"
"python.exe layout yet."
),
)
class TestSecurityInvariantsAcrossModes(unittest.TestCase):
def _run(self, code, mode):
@@ -0,0 +1,698 @@
"""Tests for execute_code env scrubbing on Windows.
On Windows the child process needs a small set of OS-essential env vars
(SYSTEMROOT, WINDIR, COMSPEC, ...) to run. Without SYSTEMROOT in particular,
``socket.socket(AF_INET, SOCK_STREAM)`` fails inside the sandbox with
WinError 10106 (Winsock can't locate mswsock.dll) and no tool call over
loopback TCP can ever succeed.
These tests cover ``_scrub_child_env`` directly so they run on every OS
the logic is conditional on a passed-in ``is_windows`` flag, not on
the host platform. We also keep a live Winsock smoke test that only runs
on a real Windows host.
Also covers the companion Windows bug: the sandbox writes
``hermes_tools.py`` and ``script.py`` into a temp dir, and those files
must be written as UTF-8 on every platform the generated stub contains
em-dash/en-dash characters in docstrings, and the default ``open(path, "w")``
on Windows uses the system locale (cp1252 typically), corrupting those
bytes. The child then fails to import with a SyntaxError:
``'utf-8' codec can't decode byte 0x97``.
"""
import os
import socket
import subprocess
import sys
import textwrap
import unittest.mock as mock
import pytest
from tools.code_execution_tool import (
_SAFE_ENV_PREFIXES,
_SECRET_SUBSTRINGS,
_WINDOWS_ESSENTIAL_ENV_VARS,
_scrub_child_env,
)
def _no_passthrough(_name):
return False
class TestWindowsEssentialAllowlist:
"""The allowlist itself — contents, shape, and invariants."""
def test_contains_winsock_required_vars(self):
# Without SYSTEMROOT the child cannot initialize Winsock.
assert "SYSTEMROOT" in _WINDOWS_ESSENTIAL_ENV_VARS
def test_contains_subprocess_required_vars(self):
# Without COMSPEC, subprocess can't resolve the default shell.
assert "COMSPEC" in _WINDOWS_ESSENTIAL_ENV_VARS
def test_contains_user_profile_vars(self):
# os.path.expanduser("~") on Windows uses USERPROFILE.
assert "USERPROFILE" in _WINDOWS_ESSENTIAL_ENV_VARS
assert "APPDATA" in _WINDOWS_ESSENTIAL_ENV_VARS
assert "LOCALAPPDATA" in _WINDOWS_ESSENTIAL_ENV_VARS
def test_contains_only_uppercase_names(self):
# Windows env var names are case-insensitive but we canonicalize to
# uppercase for the membership check (``k.upper() in _WINDOWS_...``).
for name in _WINDOWS_ESSENTIAL_ENV_VARS:
assert name == name.upper(), f"{name!r} should be uppercase"
def test_no_overlap_with_secret_substrings(self):
# Sanity: none of the essential OS vars should look like secrets.
# If this ever fires, we'd have a precedence ordering bug (secrets
# are blocked *before* the essentials check).
for name in _WINDOWS_ESSENTIAL_ENV_VARS:
assert not any(s in name for s in _SECRET_SUBSTRINGS), (
f"{name!r} looks secret-like — would be blocked before the "
"essentials allowlist can match"
)
class TestScrubChildEnvWindows:
"""Verify _scrub_child_env passes Windows essentials through when
is_windows=True and blocks them when is_windows=False (so POSIX hosts
don't inherit pointless Windows vars)."""
def _sample_windows_env(self):
"""A realistic subset of what os.environ looks like on Windows."""
return {
"SYSTEMROOT": r"C:\Windows",
"SystemDrive": "C:", # Windows preserves native case
"WINDIR": r"C:\Windows",
"ComSpec": r"C:\Windows\System32\cmd.exe",
"PATHEXT": ".COM;.EXE;.BAT;.CMD;.PY",
"USERPROFILE": r"C:\Users\alice",
"APPDATA": r"C:\Users\alice\AppData\Roaming",
"LOCALAPPDATA": r"C:\Users\alice\AppData\Local",
"PATH": r"C:\Windows\System32;C:\Python311",
"HOME": r"C:\Users\alice",
"TEMP": r"C:\Users\alice\AppData\Local\Temp",
# Should still be blocked:
"OPENAI_API_KEY": "sk-secret",
"GITHUB_TOKEN": "ghp_secret",
"MY_PASSWORD": "hunter2",
# Not matched by any rule — should be dropped on both OSes:
"RANDOM_UNKNOWN_VAR": "value",
}
def test_windows_essentials_passed_through_when_is_windows_true(self):
env = self._sample_windows_env()
scrubbed = _scrub_child_env(env,
is_passthrough=_no_passthrough,
is_windows=True)
# Every essential var from the sample env should survive.
assert scrubbed["SYSTEMROOT"] == r"C:\Windows"
assert scrubbed["SystemDrive"] == "C:" # case preserved
assert scrubbed["WINDIR"] == r"C:\Windows"
assert scrubbed["ComSpec"] == r"C:\Windows\System32\cmd.exe"
assert scrubbed["PATHEXT"] == ".COM;.EXE;.BAT;.CMD;.PY"
assert scrubbed["USERPROFILE"] == r"C:\Users\alice"
assert scrubbed["APPDATA"].endswith("Roaming")
assert scrubbed["LOCALAPPDATA"].endswith("Local")
# Safe-prefix vars still pass (baseline behavior).
assert "PATH" in scrubbed
assert "HOME" in scrubbed
assert "TEMP" in scrubbed
def test_secrets_still_blocked_on_windows(self):
"""The Windows allowlist must NOT defeat the secret-substring block.
This is the key security invariant: essentials are allowed by
*exact name*, and the secret-substring block runs before the
essentials check anyway, so a variable named e.g. ``API_KEY`` can
never sneak through just because we added Windows support.
"""
env = self._sample_windows_env()
scrubbed = _scrub_child_env(env,
is_passthrough=_no_passthrough,
is_windows=True)
assert "OPENAI_API_KEY" not in scrubbed
assert "GITHUB_TOKEN" not in scrubbed
assert "MY_PASSWORD" not in scrubbed
def test_unknown_vars_still_dropped_on_windows(self):
env = self._sample_windows_env()
scrubbed = _scrub_child_env(env,
is_passthrough=_no_passthrough,
is_windows=True)
assert "RANDOM_UNKNOWN_VAR" not in scrubbed
def test_essentials_blocked_when_is_windows_false(self):
"""On POSIX hosts, Windows-specific vars should not pass — they
have no meaning and could confuse child tooling."""
env = self._sample_windows_env()
scrubbed = _scrub_child_env(env,
is_passthrough=_no_passthrough,
is_windows=False)
# Safe prefixes still match (PATH, HOME, TEMP).
assert "PATH" in scrubbed
assert "HOME" in scrubbed
assert "TEMP" in scrubbed
# But Windows OS vars should be dropped.
assert "SYSTEMROOT" not in scrubbed
assert "WINDIR" not in scrubbed
assert "ComSpec" not in scrubbed
assert "APPDATA" not in scrubbed
def test_case_insensitive_essential_match(self):
"""Windows env var names are case-insensitive at the OS level but
Python preserves whatever case os.environ reported. The scrubber
must normalize to uppercase for the membership check."""
env = {
"SystemRoot": r"C:\Windows", # mixed case
"comspec": r"C:\Windows\System32\cmd.exe", # lowercase
"APPDATA": r"C:\Users\x\AppData\Roaming", # uppercase
}
scrubbed = _scrub_child_env(env,
is_passthrough=_no_passthrough,
is_windows=True)
assert "SystemRoot" in scrubbed
assert "comspec" in scrubbed
assert "APPDATA" in scrubbed
class TestScrubChildEnvPassthroughInteraction:
"""The passthrough hook runs *before* the secret block, so a skill
can legitimately forward a third-party API key. The Windows
essentials addition must not interfere with that."""
def test_passthrough_wins_over_secret_block(self):
env = {"TENOR_API_KEY": "x", "PATH": "/bin"}
scrubbed = _scrub_child_env(env,
is_passthrough=lambda k: k == "TENOR_API_KEY",
is_windows=False)
assert scrubbed.get("TENOR_API_KEY") == "x"
assert scrubbed.get("PATH") == "/bin"
def test_passthrough_still_works_on_windows(self):
env = {
"TENOR_API_KEY": "x",
"SYSTEMROOT": r"C:\Windows",
"OPENAI_API_KEY": "sk-secret", # not passthrough
}
scrubbed = _scrub_child_env(
env,
is_passthrough=lambda k: k == "TENOR_API_KEY",
is_windows=True,
)
assert scrubbed.get("TENOR_API_KEY") == "x"
assert scrubbed.get("SYSTEMROOT") == r"C:\Windows"
assert "OPENAI_API_KEY" not in scrubbed
@pytest.mark.skipif(
sys.platform != "win32",
reason="Winsock-specific regression — only meaningful on Windows",
)
class TestWindowsSocketSmokeTest:
"""Integration-ish smoke test: spawn a child Python with a scrubbed
env and confirm it can create an AF_INET socket. This is the
regression that motivated the fix without SYSTEMROOT the child
hits WinError 10106 before any RPC is attempted."""
def test_child_can_create_socket_with_scrubbed_env(self):
scrubbed = _scrub_child_env(os.environ, is_passthrough=_no_passthrough)
# Build a tiny child script that simply opens an AF_INET socket.
script = textwrap.dedent("""
import socket, sys
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.close()
print("OK")
sys.exit(0)
except OSError as exc:
print(f"FAIL: {exc}")
sys.exit(1)
""").strip()
result = subprocess.run(
[sys.executable, "-c", script],
env=scrubbed,
capture_output=True,
text=True,
timeout=15,
)
assert result.returncode == 0, (
f"Child failed to create socket with scrubbed env:\n"
f" stdout={result.stdout!r}\n"
f" stderr={result.stderr!r}\n"
f" scrubbed keys={sorted(scrubbed.keys())}"
)
assert "OK" in result.stdout
# ---------------------------------------------------------------------------
# POSIX equivalence guard
# ---------------------------------------------------------------------------
def _legacy_posix_scrubber(source_env, is_passthrough):
"""Verbatim copy of the pre-Windows-fix inline scrubbing logic.
This is the oracle used by TestPosixEquivalence to prove the refactor
did not change POSIX behavior. DO NOT edit this to "match" a future
production change if _scrub_child_env's POSIX behavior legitimately
needs to evolve, delete this function and adjust the equivalence test
on purpose, so the churn is visible in review.
"""
_SAFE_ENV_PREFIXES = ("PATH", "HOME", "USER", "LANG", "LC_", "TERM",
"TMPDIR", "TMP", "TEMP", "SHELL", "LOGNAME",
"XDG_", "PYTHONPATH", "VIRTUAL_ENV", "CONDA",
"HERMES_")
_SECRET_SUBSTRINGS = ("KEY", "TOKEN", "SECRET", "PASSWORD", "CREDENTIAL",
"PASSWD", "AUTH")
out = {}
for k, v in source_env.items():
if is_passthrough(k):
out[k] = v
continue
if any(s in k.upper() for s in _SECRET_SUBSTRINGS):
continue
if any(k.startswith(p) for p in _SAFE_ENV_PREFIXES):
out[k] = v
return out
class TestPosixEquivalence:
"""Lock in the invariant that _scrub_child_env(env, is_windows=False)
behaves *bit-for-bit identically* to the pre-refactor inline scrubber.
If this ever fails, it means somebody changed POSIX env-scrubbing
behavior maybe on purpose, maybe not. Either way it should land
as a deliberate, reviewed change (update _legacy_posix_scrubber
above in the same PR).
Rationale: the Windows-essentials patch refactored the scrubber into
a helper. Linux/macOS must not regress. This class gates that.
"""
_POSIX_SYNTHETIC_ENV = {
# Safe-prefix matches
"PATH": "/usr/bin:/bin",
"HOME": "/home/alice",
"USER": "alice",
"LANG": "en_US.UTF-8",
"LC_CTYPE": "en_US.UTF-8",
"TERM": "xterm-256color",
"SHELL": "/bin/zsh",
"LOGNAME": "alice",
"TMPDIR": "/tmp",
"XDG_RUNTIME_DIR": "/run/user/1000",
"XDG_CONFIG_HOME": "/home/alice/.config",
"PYTHONPATH": "/opt/lib",
"VIRTUAL_ENV": "/home/alice/.venv",
"CONDA_PREFIX": "/opt/conda",
"HERMES_HOME": "/home/alice/.hermes",
"HERMES_INTERACTIVE": "1",
# Secret-substring blocks
"OPENAI_API_KEY": "sk-xxx",
"GITHUB_TOKEN": "ghp_xxx",
"AWS_SECRET_ACCESS_KEY": "yyy",
"MY_PASSWORD": "hunter2",
# Uncategorized — must be dropped
"RANDOM_UNKNOWN": "drop-me",
"DISPLAY": ":0",
"SSH_AUTH_SOCK": "/run/user/1000/ssh-agent",
# Passthrough candidate (also matches secret block by default)
"TENOR_API_KEY": "tenor-xxx",
}
_WINDOWS_SYNTHETIC_ENV = {
# Windows-essential names (must be dropped on POSIX, passed on Win)
"SYSTEMROOT": r"C:\Windows",
"SystemDrive": "C:",
"WINDIR": r"C:\Windows",
"ComSpec": r"C:\Windows\System32\cmd.exe",
"PATHEXT": ".COM;.EXE;.BAT",
"USERPROFILE": r"C:\Users\alice",
"APPDATA": r"C:\Users\alice\AppData\Roaming",
"LOCALAPPDATA": r"C:\Users\alice\AppData\Local",
# Safe-prefix matches (cross-platform)
"PATH": r"C:\Python311;C:\Windows\System32",
"HOME": r"C:\Users\alice",
"TEMP": r"C:\Users\alice\AppData\Local\Temp",
# Secret-looking (always blocked)
"OPENAI_API_KEY": "sk-xxx",
"GITHUB_TOKEN": "ghp_xxx",
}
@pytest.mark.parametrize("env_name,env", [
("posix_synthetic", _POSIX_SYNTHETIC_ENV),
("windows_synthetic_on_posix", _WINDOWS_SYNTHETIC_ENV),
])
@pytest.mark.parametrize("pt_name,pt", [
("no_passthrough", lambda _: False),
("tenor_passthrough", lambda k: k == "TENOR_API_KEY"),
("all_passthrough", lambda _: True),
])
def test_posix_behavior_unchanged(self, env_name, env, pt_name, pt):
"""For every combination of (env shape × passthrough rule), the
new helper with is_windows=False must produce the exact same dict
as the legacy inline scrubber.
We parametrize over three passthrough rules to cover the full
surface: no passthrough, single-var passthrough (the common
skill-registered case), and everything-passes (edge case that
could expose precedence bugs)."""
expected = _legacy_posix_scrubber(env, pt)
actual = _scrub_child_env(env, is_passthrough=pt, is_windows=False)
assert actual == expected, (
f"POSIX behavior regressed for env={env_name}, passthrough={pt_name}\n"
f" only in legacy: {sorted(set(expected) - set(actual))}\n"
f" only in new: {sorted(set(actual) - set(expected))}\n"
f" value diffs: {[k for k in expected if k in actual and expected[k] != actual[k]]}"
)
def test_posix_behavior_unchanged_on_real_os_environ(self):
"""Bonus check against the actual os.environ of the host running
the test. This covers vars we might not have thought to put in
the synthetic fixtures."""
expected = _legacy_posix_scrubber(os.environ, lambda _: False)
actual = _scrub_child_env(os.environ,
is_passthrough=lambda _: False,
is_windows=False)
assert actual == expected, (
"POSIX-mode scrubber diverged from legacy behavior on real "
f"os.environ (host platform={sys.platform})"
)
def test_windows_mode_is_strict_superset_of_posix_mode(self):
"""Correctness check on the NEW behavior: is_windows=True must
keep everything POSIX mode keeps, and *may* add Windows
essentials. It must never drop a var that POSIX mode would keep
if it did, we'd have broken same-host reuse of the scrubber."""
env = {**self._POSIX_SYNTHETIC_ENV, **self._WINDOWS_SYNTHETIC_ENV}
posix_result = _scrub_child_env(env,
is_passthrough=lambda _: False,
is_windows=False)
windows_result = _scrub_child_env(env,
is_passthrough=lambda _: False,
is_windows=True)
missing = set(posix_result) - set(windows_result)
assert not missing, (
f"is_windows=True dropped vars that is_windows=False kept: {missing}"
)
# And any extras must come from the Windows essentials allowlist.
extras = set(windows_result) - set(posix_result)
for k in extras:
assert k.upper() in _WINDOWS_ESSENTIAL_ENV_VARS, (
f"Unexpected extra var in windows-mode output: {k} "
f"(not in _WINDOWS_ESSENTIAL_ENV_VARS)"
)
# ---------------------------------------------------------------------------
# UTF-8 file-write regression test
# ---------------------------------------------------------------------------
#
# The sandbox writes two Python files into a temp dir — the generated
# ``hermes_tools.py`` stub, and the LLM's ``script.py``. Both contain
# non-ASCII characters in practice: the stub has em-dashes in docstrings
# ("``tcp://host:port`` — the parent falls back..."), and user scripts
# routinely contain non-ASCII strings, comments, or Unicode identifiers.
#
# On Windows, ``open(path, "w")`` without encoding= uses the system locale
# (cp1252 on US/UK installs), which cannot encode em-dashes. Python then
# tries to decode the file as UTF-8 when importing it (PEP 3120), fails,
# and the sandbox aborts with:
#
# SyntaxError: (unicode error) 'utf-8' codec can't decode byte 0x97
# in position N: invalid start byte
#
# This was the *second* Windows-specific bug (WinError 10106 was the first).
# The fix is to always pass ``encoding="utf-8"`` when writing Python source.
class TestSandboxWritesUtf8:
"""Verify the file-write call sites use UTF-8 explicitly, not the
platform default. We check the source of ``execute_code`` rather
than spawning a real sandbox because the latter needs a full agent
context but the code inspection is deterministic and fast."""
def test_stub_and_script_writes_specify_utf8(self):
"""Both ``hermes_tools.py`` and ``script.py`` writes in
``_execute_local`` must pass ``encoding="utf-8"``."""
import tools.code_execution_tool as cet
src = open(cet.__file__, encoding="utf-8").read()
# There should be no ``open(path, "w")`` without encoding= for
# the two staging files. Grep-style check: find every write of
# a .py file inside tmpdir and assert the line also contains
# ``encoding="utf-8"`` within a short window.
import re
pattern = re.compile(
r'open\(\s*os\.path\.join\(\s*tmpdir\s*,\s*"[^"]+\.py"\s*\)\s*,\s*"w"[^)]*\)'
)
for match in pattern.finditer(src):
line = match.group(0)
assert 'encoding="utf-8"' in line or "encoding='utf-8'" in line, (
f"Sandbox file write missing encoding=\"utf-8\" on Windows: {line!r}"
)
def test_file_rpc_stub_uses_utf8(self):
"""The file-based RPC transport stub (used by remote backends)
reads/writes JSON response files. Those must also specify UTF-8
so non-ASCII tool results survive the round-trip intact."""
from tools.code_execution_tool import generate_hermes_tools_module
stub = generate_hermes_tools_module(["terminal"], transport="file")
# The generated stub should open response + request files as UTF-8.
assert 'encoding="utf-8"' in stub, (
"File-based RPC stub does not specify encoding=\"utf-8\""
"will corrupt non-ASCII tool results on non-UTF-8 locales."
)
def test_stub_source_roundtrips_through_utf8(self):
"""Concrete regression: write the generated stub to a temp file
using ``encoding="utf-8"``, then parse it. This is what the
sandbox does, and it must succeed even when the stub contains
em-dashes (which it does check the transport-header docstring).
"""
from tools.code_execution_tool import generate_hermes_tools_module
import tempfile, ast
stub = generate_hermes_tools_module(
["terminal", "read_file", "write_file"], transport="uds"
)
# Sanity: stub actually contains a non-ASCII character, otherwise
# this test wouldn't prove anything meaningful.
non_ascii = [c for c in stub if ord(c) > 127]
assert non_ascii, (
"Generated stub is pure ASCII — test is meaningless. If the "
"stub's docstrings have lost their em-dashes, update this "
"assertion, but be aware the original regression is no longer "
"covered."
)
with tempfile.NamedTemporaryFile(
mode="w", suffix=".py", delete=False, encoding="utf-8"
) as f:
f.write(stub)
tmp_path = f.name
try:
# Re-read and parse exactly like the child Python would.
with open(tmp_path, encoding="utf-8") as fh:
round_tripped = fh.read()
assert round_tripped == stub, "UTF-8 round-trip corrupted the stub"
ast.parse(round_tripped) # must not raise SyntaxError
finally:
os.unlink(tmp_path)
@pytest.mark.skipif(
sys.platform != "win32",
reason="cp1252 default-encoding regression is Windows-specific",
)
def test_windows_default_encoding_would_have_failed(self):
"""Negative control: prove that on Windows, writing the stub
*without* ``encoding="utf-8"`` would corrupt the file. If this
test ever starts failing (i.e. default write succeeds), it means
Python's default encoding has changed and the explicit UTF-8
requirement may be obsolete reconsider the fix."""
from tools.code_execution_tool import generate_hermes_tools_module
import tempfile
stub = generate_hermes_tools_module(["terminal"], transport="uds")
# Find a non-ASCII character we can use to prove the corruption.
non_ascii = [c for c in stub if ord(c) > 127]
if not non_ascii:
pytest.skip("stub has no non-ASCII chars — nothing to corrupt")
# Write with default encoding (simulating the old buggy code).
with tempfile.NamedTemporaryFile(
mode="w", suffix=".py", delete=False
) as f:
try:
f.write(stub)
tmp_path = f.name
wrote_successfully = True
except UnicodeEncodeError:
# Default encoding can't even encode it — that's the bug
# in a different form. Still proves the point.
tmp_path = f.name
wrote_successfully = False
try:
if not wrote_successfully:
# Default-encoding write raised outright. The bug is real.
return
# Read back as UTF-8 (what Python does on import).
with open(tmp_path, encoding="utf-8") as fh:
try:
fh.read()
# If this succeeds on Windows, the platform default is
# already UTF-8 (e.g. Python 3.15 with UTF-8 mode on).
# In that case the explicit encoding= is belt-and-
# suspenders but no longer strictly required. Skip.
pytest.skip(
"Default text-file encoding is UTF-8-compatible on "
"this Windows build — explicit encoding= is no "
"longer load-bearing, but keep it for belt-and-"
"suspenders."
)
except UnicodeDecodeError:
# Exactly the failure mode that motivated the fix.
pass
finally:
os.unlink(tmp_path)
# ---------------------------------------------------------------------------
# UTF-8 stdio regression test
# ---------------------------------------------------------------------------
#
# The third Windows-specific sandbox bug: after the UTF-8 file-write fix
# let the child import hermes_tools, a user script that printed non-ASCII
# to stdout still crashed with:
#
# UnicodeEncodeError: 'charmap' codec can't encode character '\u2192'
# in position N: character maps to <undefined>
#
# Python's sys.stdout on Windows is bound to the console code page
# (cp1252 on US-locale installs) when the process is attached to a pipe
# without PYTHONIOENCODING set. LLM-generated scripts routinely print
# em-dashes, arrows, accented chars, emoji — all of which break.
#
# Fix: spawn the child with PYTHONIOENCODING=utf-8 and PYTHONUTF8=1.
# The latter also makes open()'s default encoding UTF-8 (PEP 540),
# belt-and-suspenders for user scripts that do their own file I/O.
class TestChildStdioIsUtf8:
"""Verify the sandbox child is spawned with UTF-8 stdio encoding,
so LLM scripts can print non-ASCII without crashing on Windows."""
def test_popen_env_sets_pythonioencoding_utf8(self):
"""Source-level check: the Popen call site must set
PYTHONIOENCODING=utf-8 in child_env."""
import tools.code_execution_tool as cet
src = open(cet.__file__, encoding="utf-8").read()
assert 'child_env["PYTHONIOENCODING"] = "utf-8"' in src, (
"PYTHONIOENCODING=utf-8 missing from child env — Windows "
"scripts that print non-ASCII will crash with "
"UnicodeEncodeError."
)
def test_popen_env_sets_pythonutf8_mode(self):
"""Source-level check: PYTHONUTF8=1 must be set too — it makes
open()'s default encoding UTF-8 in user-written file I/O."""
import tools.code_execution_tool as cet
src = open(cet.__file__, encoding="utf-8").read()
assert 'child_env["PYTHONUTF8"] = "1"' in src, (
"PYTHONUTF8=1 missing from child env — user scripts that "
"call open(path, 'w') without encoding= will produce "
"locale-encoded files on Windows."
)
def test_live_child_can_print_non_ascii(self):
"""Live regression: spawn a Python child with the same env
treatment the sandbox uses (PYTHONIOENCODING=utf-8 + PYTHONUTF8=1)
and verify it can print em-dashes, arrows, and emoji to stdout
without crashing. This is the exact scenario that broke in live
usage.
Runs on every OS on POSIX the fix is belt-and-suspenders but
still load-bearing for C.ASCII locale environments.
"""
script = textwrap.dedent("""
import sys
# Mix of chars that cp1252 can't encode: arrow, emoji.
print("em-dash \\u2014 arrow \\u2192 emoji \\U0001f680")
sys.exit(0)
""").strip()
# Build a scrubbed env the same way the sandbox does, then apply
# the stdio overrides.
scrubbed = _scrub_child_env(os.environ, is_passthrough=_no_passthrough)
scrubbed["PYTHONIOENCODING"] = "utf-8"
scrubbed["PYTHONUTF8"] = "1"
result = subprocess.run(
[sys.executable, "-c", script],
env=scrubbed,
capture_output=True,
timeout=15,
# Don't decode at the subprocess boundary — we want to check
# the raw bytes match UTF-8, same as what the sandbox does.
)
assert result.returncode == 0, (
f"Child crashed printing non-ASCII:\n"
f" stdout (raw): {result.stdout!r}\n"
f" stderr (raw): {result.stderr!r}"
)
decoded = result.stdout.decode("utf-8")
assert "\u2014" in decoded, f"em-dash missing from output: {decoded!r}"
assert "\u2192" in decoded, f"arrow missing from output: {decoded!r}"
assert "\U0001f680" in decoded, f"emoji missing from output: {decoded!r}"
@pytest.mark.skipif(
sys.platform != "win32",
reason="cp1252 stdout default is Windows-specific",
)
def test_windows_child_without_utf8_env_would_fail(self):
"""Negative control: spawn a Python child *without* our env
overrides and prove that on Windows, printing non-ASCII fails.
If this ever starts passing, Python has changed its default
stdio encoding on Windows and the fix may be obsolete but
keep the env vars anyway for belt-and-suspenders."""
script = textwrap.dedent("""
import sys
print("em-dash \\u2014 arrow \\u2192")
sys.exit(0)
""").strip()
# Scrubbed env WITHOUT the PYTHONIOENCODING / PYTHONUTF8 overrides.
# Also scrub PYTHONUTF8 and PYTHONIOENCODING from the inherited
# env so we reproduce the buggy state even if the parent test
# runner has them set.
scrubbed = _scrub_child_env(os.environ, is_passthrough=_no_passthrough)
for k in ("PYTHONIOENCODING", "PYTHONUTF8", "PYTHONLEGACYWINDOWSSTDIO"):
scrubbed.pop(k, None)
result = subprocess.run(
[sys.executable, "-c", script],
env=scrubbed,
capture_output=True,
text=False,
timeout=15,
)
# Either the child crashed (expected), or modern Python handled
# it anyway — in which case the fix is still defensive but no
# longer strictly required. Skip with a note if so.
if result.returncode == 0 and b"\xe2\x80\x94" in result.stdout:
pytest.skip(
"This Python/Windows build handles non-ASCII stdout even "
"without PYTHONIOENCODING/PYTHONUTF8 — fix is defensive "
"but no longer strictly load-bearing. Keep the env vars "
"for older Python builds and C.ASCII-locale containers."
)
# Otherwise: crash OR garbled output — both count as proving the
# bug is real on this system.
+812
View File
@@ -0,0 +1,812 @@
"""Behavioral tests for Windows-specific compatibility fixes.
Complements ``tests/tools/test_windows_compat.py`` (which does source-level
pattern linting) with cross-platform-mocked tests that exercise the actual
code paths Hermes takes on native Windows.
Runs on Linux CI every test mocks ``sys.platform``, ``subprocess.run``,
and ``os.kill`` as needed to simulate Windows behavior without requiring a
Windows runner.
"""
from __future__ import annotations
import importlib
import os
import signal
import subprocess
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
# ---------------------------------------------------------------------------
# configure_windows_stdio
# ---------------------------------------------------------------------------
class TestConfigureWindowsStdio:
"""``hermes_cli.stdio.configure_windows_stdio`` wiring.
The function must:
- be a no-op on non-Windows
- only configure once per process (idempotent)
- set PYTHONIOENCODING / PYTHONUTF8 without overriding explicit user settings
- reconfigure sys.stdout/stderr/stdin to UTF-8 on Windows
- flip the console code page to CP_UTF8 (65001) via ctypes
- respect HERMES_DISABLE_WINDOWS_UTF8 opt-out
"""
@pytest.fixture(autouse=True)
def _reset_configured(self, monkeypatch):
"""Reload the module before each test so the _CONFIGURED flag resets."""
# Remove from sys.modules so import triggers a fresh load
sys.modules.pop("hermes_cli.stdio", None)
# Fresh import now; tests import from hermes_cli.stdio themselves,
# but this guarantees the module they get is a brand-new copy.
import hermes_cli.stdio as _s
_s._CONFIGURED = False
yield
sys.modules.pop("hermes_cli.stdio", None)
def test_no_op_on_posix(self):
from hermes_cli import stdio
assert stdio.is_windows() is False
result = stdio.configure_windows_stdio()
assert result is False
def test_idempotent(self):
from hermes_cli import stdio
stdio.configure_windows_stdio()
# Second call returns False because _CONFIGURED is set
assert stdio.configure_windows_stdio() is False
def test_windows_path_sets_env_and_reconfigures_streams(self, monkeypatch):
from hermes_cli import stdio
monkeypatch.setattr(stdio, "is_windows", lambda: True)
# Pretend the user has no prior setting
monkeypatch.delenv("PYTHONIOENCODING", raising=False)
monkeypatch.delenv("PYTHONUTF8", raising=False)
monkeypatch.delenv("HERMES_DISABLE_WINDOWS_UTF8", raising=False)
monkeypatch.delenv("EDITOR", raising=False)
monkeypatch.delenv("VISUAL", raising=False)
reconfigure_calls = []
def fake_reconfigure(stream, *, encoding="utf-8", errors="replace"):
reconfigure_calls.append((stream, encoding, errors))
cp_calls = []
def fake_flip():
cp_calls.append(True)
monkeypatch.setattr(stdio, "_reconfigure_stream", fake_reconfigure)
monkeypatch.setattr(stdio, "_flip_console_code_page_to_utf8", fake_flip)
# Pretend notepad.exe is on PATH (it always is on real Windows hosts,
# but not on the Linux CI runner — mock it so the editor default
# survives).
monkeypatch.setattr(stdio, "_default_windows_editor", lambda: "notepad")
result = stdio.configure_windows_stdio()
assert result is True
assert os.environ.get("PYTHONIOENCODING") == "utf-8"
assert os.environ.get("PYTHONUTF8") == "1"
# EDITOR must be set so prompt_toolkit's open_in_editor finds
# a working program on Windows (it defaults to /usr/bin/nano).
assert os.environ.get("EDITOR") == "notepad"
assert len(cp_calls) == 1 # SetConsoleOutputCP path hit
assert len(reconfigure_calls) == 3 # stdout, stderr, stdin
def test_respects_existing_editor_var(self, monkeypatch):
"""User's explicit EDITOR wins over our default."""
from hermes_cli import stdio
monkeypatch.setattr(stdio, "is_windows", lambda: True)
monkeypatch.setenv("EDITOR", "code --wait")
monkeypatch.setattr(stdio, "_reconfigure_stream", lambda *a, **kw: None)
monkeypatch.setattr(stdio, "_flip_console_code_page_to_utf8", lambda: None)
monkeypatch.setattr(stdio, "_default_windows_editor", lambda: "notepad")
stdio.configure_windows_stdio()
assert os.environ["EDITOR"] == "code --wait"
def test_respects_existing_visual_var(self, monkeypatch):
"""VISUAL takes precedence over our EDITOR default too."""
from hermes_cli import stdio
monkeypatch.setattr(stdio, "is_windows", lambda: True)
monkeypatch.delenv("EDITOR", raising=False)
monkeypatch.setenv("VISUAL", "nvim")
monkeypatch.setattr(stdio, "_reconfigure_stream", lambda *a, **kw: None)
monkeypatch.setattr(stdio, "_flip_console_code_page_to_utf8", lambda: None)
monkeypatch.setattr(stdio, "_default_windows_editor", lambda: "notepad")
stdio.configure_windows_stdio()
# EDITOR should NOT be set when VISUAL already is (prompt_toolkit
# checks VISUAL first anyway, but we also shouldn't override it).
assert os.environ.get("EDITOR", "") != "notepad"
assert os.environ["VISUAL"] == "nvim"
def test_respects_existing_env_var(self, monkeypatch):
"""User's explicit PYTHONIOENCODING wins over our default."""
from hermes_cli import stdio
monkeypatch.setattr(stdio, "is_windows", lambda: True)
monkeypatch.setenv("PYTHONIOENCODING", "latin-1")
monkeypatch.setattr(stdio, "_reconfigure_stream", lambda *a, **kw: None)
monkeypatch.setattr(stdio, "_flip_console_code_page_to_utf8", lambda: None)
stdio.configure_windows_stdio()
assert os.environ["PYTHONIOENCODING"] == "latin-1"
@pytest.mark.parametrize("optout", ["1", "true", "True", "yes"])
def test_disable_flag_short_circuits(self, monkeypatch, optout):
from hermes_cli import stdio
monkeypatch.setattr(stdio, "is_windows", lambda: True)
monkeypatch.setenv("HERMES_DISABLE_WINDOWS_UTF8", optout)
reconfigure_hit = []
monkeypatch.setattr(
stdio,
"_reconfigure_stream",
lambda *a, **kw: reconfigure_hit.append(True),
)
result = stdio.configure_windows_stdio()
assert result is False
assert reconfigure_hit == [], "opt-out must skip stream reconfiguration"
def test_reconfigure_stream_handles_missing_method(self, monkeypatch):
"""StringIO-like objects without .reconfigure() must not blow up."""
from hermes_cli import stdio
import io
buf = io.StringIO()
# Must not raise
stdio._reconfigure_stream(buf)
# ---------------------------------------------------------------------------
# terminate_pid — the centralized kill primitive
# ---------------------------------------------------------------------------
class TestTerminatePidRoutingOnWindows:
"""``gateway.status.terminate_pid`` must use taskkill /T /F on Windows.
On Linux we can't reload gateway/status with sys.platform=win32 because
the module unconditionally imports ``msvcrt`` in that branch. Instead
we patch the module-level ``_IS_WINDOWS`` flag and ``subprocess.run``
on the already-loaded module, which exercises the same branching code.
"""
def test_force_uses_taskkill_on_windows(self, monkeypatch):
from gateway import status
captured = {}
def fake_run(args, **kwargs):
captured["args"] = args
result = MagicMock()
result.returncode = 0
result.stderr = ""
result.stdout = ""
return result
monkeypatch.setattr(status, "_IS_WINDOWS", True)
monkeypatch.setattr(status.subprocess, "run", fake_run)
status.terminate_pid(12345, force=True)
assert captured["args"][0] == "taskkill"
assert "/PID" in captured["args"]
assert "12345" in captured["args"]
assert "/T" in captured["args"]
assert "/F" in captured["args"]
def test_force_taskkill_failure_raises_oserror(self, monkeypatch):
from gateway import status
def fake_run(args, **kwargs):
result = MagicMock()
result.returncode = 128
result.stderr = "ERROR: The process cannot be terminated."
result.stdout = ""
return result
monkeypatch.setattr(status, "_IS_WINDOWS", True)
monkeypatch.setattr(status.subprocess, "run", fake_run)
with pytest.raises(OSError, match="cannot be terminated"):
status.terminate_pid(12345, force=True)
def test_graceful_on_windows_uses_os_kill_sigterm(self, monkeypatch):
"""Non-force path calls os.kill with SIGTERM (Windows has no SIGKILL).
``terminate_pid(pid)`` with force=False bypasses the taskkill branch
and uses ``os.kill`` directly so platform doesn't actually matter
for the signal choice. Verifies the getattr fallback works.
"""
from gateway import status
captured = {}
def fake_kill(pid, sig):
captured["pid"] = pid
captured["sig"] = sig
monkeypatch.setattr(status.os, "kill", fake_kill)
status.terminate_pid(99, force=False)
assert captured["pid"] == 99
assert captured["sig"] == signal.SIGTERM
def test_taskkill_not_found_falls_back_to_os_kill(self, monkeypatch):
"""On Windows without taskkill (WinPE, containers), fall back gracefully."""
from gateway import status
captured = {}
def fake_run(args, **kwargs):
raise FileNotFoundError(2, "taskkill not found")
def fake_kill(pid, sig):
captured["pid"] = pid
captured["sig"] = sig
monkeypatch.setattr(status, "_IS_WINDOWS", True)
monkeypatch.setattr(status.subprocess, "run", fake_run)
monkeypatch.setattr(status.os, "kill", fake_kill)
status.terminate_pid(42, force=True)
assert captured["pid"] == 42
assert captured["sig"] == signal.SIGTERM
# ---------------------------------------------------------------------------
# SIGKILL fallback pattern
# ---------------------------------------------------------------------------
class TestSigkillFallback:
"""Modules that want SIGKILL must fall back to SIGTERM when absent."""
def test_getattr_fallback_works_when_sigkill_missing(self, monkeypatch):
"""The `getattr(signal, "SIGKILL", signal.SIGTERM)` pattern."""
# Build a stand-in signal module with no SIGKILL attribute
fake_signal = MagicMock()
del fake_signal.SIGKILL # ensure it's absent
fake_signal.SIGTERM = 15
result = getattr(fake_signal, "SIGKILL", fake_signal.SIGTERM)
assert result == 15
def test_getattr_fallback_prefers_sigkill_when_present(self):
"""On POSIX the fallback is a no-op: real SIGKILL wins."""
result = getattr(signal, "SIGKILL", signal.SIGTERM)
assert result == signal.SIGKILL
@pytest.mark.parametrize(
"module_path, line_pattern",
[
("hermes_cli.kanban_db", 'getattr(signal, "SIGKILL", signal.SIGTERM)'),
],
)
def test_module_uses_getattr_fallback(self, module_path, line_pattern):
"""Source-level check that our modules use the safe fallback."""
rel = module_path.replace(".", "/") + ".py"
root = Path(__file__).resolve().parents[2]
source = (root / rel).read_text(encoding="utf-8")
assert line_pattern in source, (
f"{rel} must use the getattr fallback pattern on its SIGKILL site"
)
# ---------------------------------------------------------------------------
# OSError widening on os.kill(pid, 0) probes
# ---------------------------------------------------------------------------
class TestProcessRegistryOSErrorWidening:
"""_is_host_pid_alive must treat Windows' OSError as 'not alive'."""
def test_oserror_treated_as_not_alive(self, monkeypatch):
from tools.process_registry import ProcessRegistry
def fake_kill(pid, sig):
# Simulate Windows' WinError 87 for an unknown PID
raise OSError(22, "Invalid argument")
monkeypatch.setattr("tools.process_registry.os.kill", fake_kill)
assert ProcessRegistry._is_host_pid_alive(12345) is False
def test_permission_error_treated_as_not_alive(self, monkeypatch):
"""Conservative: PermissionError also means 'not alive' (matches existing behavior)."""
from tools.process_registry import ProcessRegistry
def fake_kill(pid, sig):
raise PermissionError(1, "Operation not permitted")
monkeypatch.setattr("tools.process_registry.os.kill", fake_kill)
assert ProcessRegistry._is_host_pid_alive(12345) is False
def test_zero_or_none_pid_returns_false_without_calling_kill(self, monkeypatch):
"""No wasted syscall on falsy pids."""
from tools.process_registry import ProcessRegistry
kill_calls = []
monkeypatch.setattr(
"tools.process_registry.os.kill",
lambda pid, sig: kill_calls.append(pid),
)
assert ProcessRegistry._is_host_pid_alive(None) is False
assert ProcessRegistry._is_host_pid_alive(0) is False
assert kill_calls == []
def test_alive_pid_returns_true(self, monkeypatch):
from tools.process_registry import ProcessRegistry
# os.kill returning None (default) means "probe succeeded → pid alive"
monkeypatch.setattr("tools.process_registry.os.kill", lambda pid, sig: None)
assert ProcessRegistry._is_host_pid_alive(os.getpid()) is True
# ---------------------------------------------------------------------------
# tzdata dependency
# ---------------------------------------------------------------------------
class TestTzdataDependencyDeclared:
"""Windows installs must pull tzdata for zoneinfo to work."""
def test_pyproject_declares_tzdata_for_win32(self):
root = Path(__file__).resolve().parents[2]
source = (root / "pyproject.toml").read_text(encoding="utf-8")
# The dependency line should be conditional on sys_platform == 'win32'
# and should NOT be in the core dependencies for Linux/macOS.
assert (
'tzdata>=2023.3; sys_platform == \'win32\'' in source
or "tzdata>=2023.3; sys_platform == 'win32'" in source
or 'tzdata>=2023.3; sys_platform == "win32"' in source
), "tzdata must be a Windows-only dep in pyproject.toml dependencies"
# ---------------------------------------------------------------------------
# README / docs consistency
# ---------------------------------------------------------------------------
class TestReadmeNoLongerSaysWindowsUnsupported:
"""The README shouldn't claim native Windows isn't supported."""
def test_readme_does_not_say_not_supported(self):
root = Path(__file__).resolve().parents[2]
source = (root / "README.md").read_text(encoding="utf-8")
# Previous string (removed in this PR): "Native Windows is not supported"
assert "Native Windows is not supported" not in source, (
"README.md still says native Windows is not supported — update the "
"install copy to reflect the PowerShell installer."
)
def test_readme_mentions_powershell_installer(self):
root = Path(__file__).resolve().parents[2]
source = (root / "README.md").read_text(encoding="utf-8")
assert "install.ps1" in source, (
"README.md must point at scripts/install.ps1 for Windows users"
)
# ---------------------------------------------------------------------------
# pty_bridge graceful import on Windows
# ---------------------------------------------------------------------------
class TestWebServerPtyBridgeGuard:
"""The web server must not crash if pty_bridge can't import (Windows)."""
def test_import_guard_present_in_source(self):
root = Path(__file__).resolve().parents[2]
source = (root / "hermes_cli" / "web_server.py").read_text(encoding="utf-8")
assert "_PTY_BRIDGE_AVAILABLE" in source
assert "except ImportError" in source, (
"web_server.py must wrap the pty_bridge import in try/except ImportError"
)
def test_pty_handler_checks_availability_flag(self):
"""The /api/pty handler must short-circuit when the bridge is unavailable."""
root = Path(__file__).resolve().parents[2]
source = (root / "hermes_cli" / "web_server.py").read_text(encoding="utf-8")
assert "if not _PTY_BRIDGE_AVAILABLE" in source, (
"/api/pty handler must return a friendly error when PTY is unavailable"
)
# ---------------------------------------------------------------------------
# Entry points wire configure_windows_stdio
# ---------------------------------------------------------------------------
class TestEntryPointsConfigureStdio:
"""cli.py, hermes_cli/main.py, gateway/run.py must call configure_windows_stdio."""
@pytest.mark.parametrize(
"relpath",
["cli.py", "hermes_cli/main.py", "gateway/run.py"],
)
def test_entry_point_calls_configure_stdio(self, relpath):
root = Path(__file__).resolve().parents[2]
source = (root / relpath).read_text(encoding="utf-8")
assert "configure_windows_stdio" in source, (
f"{relpath} must call hermes_cli.stdio.configure_windows_stdio() "
"early in startup so Windows consoles render Unicode without crashing"
)
# ---------------------------------------------------------------------------
# _subprocess_compat shared helpers
# ---------------------------------------------------------------------------
class TestSubprocessCompatHelpers:
"""hermes_cli/_subprocess_compat.py POSIX + Windows behaviour."""
def test_is_windows_matches_sys_platform(self):
from hermes_cli import _subprocess_compat as sc
assert sc.IS_WINDOWS == (sys.platform == "win32")
def test_resolve_node_command_returns_absolute_on_posix(self):
"""On Linux, resolve_node_command('sh', ['-c','echo hi']) picks up /bin/sh."""
from hermes_cli._subprocess_compat import resolve_node_command
# We can't assert "npm is on PATH" portably; use `sh` which is
# guaranteed on POSIX. On Windows the test only confirms the
# no-crash fallback path.
argv = resolve_node_command("sh", ["-c", "echo hi"])
assert argv[1:] == ["-c", "echo hi"]
# First element is either an absolute path (sh found) or the bare
# name (fallback) — both are acceptable behaviours.
def test_resolve_node_command_fallback_when_absent(self):
from hermes_cli._subprocess_compat import resolve_node_command
argv = resolve_node_command(
"zzz-definitely-not-on-path-xyzzy", ["--help"]
)
# Must fall back to the bare name — NOT return None, NOT crash.
assert argv[0] == "zzz-definitely-not-on-path-xyzzy"
assert argv[1:] == ["--help"]
def test_windows_flags_zero_on_posix(self):
from hermes_cli._subprocess_compat import (
windows_detach_flags,
windows_hide_flags,
)
if sys.platform != "win32":
assert windows_detach_flags() == 0
assert windows_hide_flags() == 0
def test_windows_detach_popen_kwargs_is_posix_equivalent_on_posix(self):
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
kwargs = windows_detach_popen_kwargs()
if sys.platform != "win32":
# POSIX path MUST produce start_new_session=True, which maps to
# os.setsid() in the child — identical to the unchanged main
# branch behaviour. Do NOT break Linux/macOS here.
assert kwargs == {"start_new_session": True}
else:
# Windows path must include creationflags with all 3 bits set.
assert "creationflags" in kwargs
assert kwargs["creationflags"] != 0
# No start_new_session on Windows (silently no-op there).
assert "start_new_session" not in kwargs
def test_windows_detach_flags_has_expected_win32_bits(self, monkeypatch):
"""Simulate Windows to verify flag bundle."""
from hermes_cli import _subprocess_compat as sc
monkeypatch.setattr(sc, "IS_WINDOWS", True)
flags = sc.windows_detach_flags()
# CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS | CREATE_NO_WINDOW
assert flags & 0x00000200, "missing CREATE_NEW_PROCESS_GROUP"
assert flags & 0x00000008, "missing DETACHED_PROCESS"
assert flags & 0x08000000, "missing CREATE_NO_WINDOW"
# ---------------------------------------------------------------------------
# tui_gateway/entry.py signal installation survives absent POSIX signals
# ---------------------------------------------------------------------------
class TestTuiGatewayEntrySignalGuards:
"""Importing tui_gateway.entry must not crash when SIGPIPE/SIGHUP absent.
Linux has both signals, so this is mostly a source-level invariant check
(no bare ``signal.SIGPIPE`` at module level without a ``hasattr`` guard).
On Windows the import would have raised AttributeError before this fix.
"""
def test_source_guards_each_signal_installation(self):
root = Path(__file__).resolve().parents[2]
source = (root / "tui_gateway" / "entry.py").read_text(encoding="utf-8")
# Every signal.signal(...) at module scope must be preceded by a
# hasattr check. We look at the text: no bare "signal.signal("
# call should appear outside a function body without a guard.
# Simpler heuristic: all SIGPIPE / SIGHUP references outside the
# dict-building loop must be wrapped in hasattr.
assert 'hasattr(signal, "SIGPIPE")' in source
assert 'hasattr(signal, "SIGHUP")' in source
assert 'hasattr(signal, "SIGTERM")' in source
assert 'hasattr(signal, "SIGINT")' in source
def test_module_imports_cleanly(self):
"""Importing the module must not raise — verifies the guards work."""
# Drop any cached import so the module re-initialises
for mod in list(sys.modules):
if mod.startswith("tui_gateway"):
del sys.modules[mod]
import tui_gateway.entry # noqa: F401 # must not raise
# ---------------------------------------------------------------------------
# hermes_cli/kanban_db.py waitpid guard
# ---------------------------------------------------------------------------
class TestKanbanWaitpidWindowsGuard:
"""os.WNOHANG doesn't exist on Windows — the dispatcher tick reap loop
must be gated behind ``os.name != "nt"``."""
def test_source_gates_waitpid_loop(self):
root = Path(__file__).resolve().parents[2]
source = (root / "hermes_cli" / "kanban_db.py").read_text(encoding="utf-8")
# Find the waitpid call and confirm it's inside a POSIX gate.
idx = source.find("os.waitpid(-1, os.WNOHANG)")
assert idx > 0, "waitpid call must exist"
# Look backwards up to 400 chars for the gate.
preamble = source[max(0, idx - 400):idx]
assert 'os.name != "nt"' in preamble or "os.name != 'nt'" in preamble, (
"os.waitpid(-1, os.WNOHANG) must sit behind an os.name != 'nt' guard"
)
# ---------------------------------------------------------------------------
# code_execution_tool TCP loopback on Windows
# ---------------------------------------------------------------------------
class TestCodeExecutionTransportTcpFallback:
"""The RPC transport must fall back to TCP on Windows.
We can't easily execute the sandbox on Linux CI in Windows mode, but we
CAN assert that the generated client module supports both AF_UNIX and
AF_INET endpoints based on the HERMES_RPC_SOCKET format.
"""
def test_generated_client_handles_tcp_endpoint(self):
root = Path(__file__).resolve().parents[2]
source = (root / "tools" / "code_execution_tool.py").read_text(encoding="utf-8")
# _UDS_TRANSPORT_HEADER body must parse both transports.
assert 'endpoint.startswith("tcp://")' in source, (
"generated sandbox client must accept tcp:// endpoints for Windows"
)
assert "socket.AF_INET" in source, (
"generated sandbox client must be able to open AF_INET sockets"
)
def test_server_side_branches_on_use_tcp_rpc(self):
root = Path(__file__).resolve().parents[2]
source = (root / "tools" / "code_execution_tool.py").read_text(encoding="utf-8")
assert "_use_tcp_rpc = _IS_WINDOWS" in source
assert 'rpc_endpoint = f"tcp://{_host}:{_port}"' in source
# ---------------------------------------------------------------------------
# cron/scheduler.py /bin/bash dynamic resolution
# ---------------------------------------------------------------------------
class TestCronSchedulerBashResolution:
"""cron.scheduler must NOT hardcode /bin/bash — .sh scripts need a
dynamically-resolved bash so Windows (Git Bash) works."""
def test_source_uses_shutil_which_for_bash(self):
root = Path(__file__).resolve().parents[2]
source = (root / "cron" / "scheduler.py").read_text(encoding="utf-8")
# The old hardcoded path should be gone as the sole bash source.
# It may still appear as a POSIX fallback after shutil.which(), so
# we check for the shutil.which call near the .sh/.bash branch.
assert 'shutil.which("bash")' in source, (
"cron.scheduler must resolve bash dynamically via shutil.which"
)
def test_error_message_when_bash_missing(self):
root = Path(__file__).resolve().parents[2]
source = (root / "cron" / "scheduler.py").read_text(encoding="utf-8")
# The graceful-failure message must mention "bash not found" so
# Windows users without Git Bash see an actionable error instead
# of a WinError 2 traceback.
assert "bash not found" in source.lower()
# ---------------------------------------------------------------------------
# Node-ecosystem launcher resolution (npm / npx / node)
# ---------------------------------------------------------------------------
class TestNpmBareSpawnsResolved:
"""Every spawn site that launches ``npm``/``npx`` must resolve via
shutil.which / hermes_cli._subprocess_compat.resolve_node_command
so Windows can execute the .cmd batch shims."""
@pytest.mark.parametrize(
"relpath",
[
"hermes_cli/tools_config.py",
"hermes_cli/doctor.py",
"gateway/platforms/whatsapp.py",
"tools/browser_tool.py",
],
)
def test_no_bare_npm_or_npx_in_popen_argv(self, relpath):
"""Reject ``subprocess.run(["npm", ...])`` / ``["npx", ...]`` patterns.
Those fail on Windows with WinError 193. Callers must resolve
via shutil.which(...) and pass the absolute path (or fall back
to the bare name only as a last resort behind a variable).
"""
root = Path(__file__).resolve().parents[2]
source = (root / relpath).read_text(encoding="utf-8")
# The forbidden literal: a subprocess invocation that names npm
# or npx as a bare string inside an argv list.
forbidden_patterns = [
'["npm",',
'["npx",',
"['npm',",
"['npx',",
]
for pat in forbidden_patterns:
# Exception: strings inside error-message text or comments are fine.
# We only fail if the literal appears in an argv position, which
# we approximate by checking it isn't inside a print/log/comment.
# Find all occurrences and verify they're behind shutil.which.
idx = 0
while True:
idx = source.find(pat, idx)
if idx < 0:
break
# Look at the preceding 120 chars — if "shutil.which" appears
# there, or the pattern is inside a comment/string, it's fine.
context = source[max(0, idx - 120):idx]
if "#" in context.split("\n")[-1]:
idx += len(pat)
continue
# Argv forms that START with a bare npm/npx are the bug.
raise AssertionError(
f"{relpath}: bare {pat!r} still present at offset {idx}"
f"resolve via shutil.which(...) so Windows can execute .cmd shims"
)
# ---------------------------------------------------------------------------
# tools/environments/local.py Windows temp dir & PATH injection
# ---------------------------------------------------------------------------
class TestLocalEnvironmentWindowsTempDir:
"""LocalEnvironment.get_temp_dir must return a native Windows path on
Windows, NOT the POSIX ``/tmp`` literal (which Python can't open)."""
def test_posix_path_preserved_on_linux(self):
"""Linux/macOS behaviour MUST be unchanged — return / tmp or
tempfile.gettempdir()-derived POSIX path. This is the 'do no harm'
test regressions here break every Unix user's terminal tool."""
from tools.environments.local import LocalEnvironment
env = LocalEnvironment(cwd="/tmp", timeout=10, env={})
tmp_dir = env.get_temp_dir()
if sys.platform != "win32":
assert tmp_dir.startswith("/"), (
f"POSIX temp dir must start with '/'; got {tmp_dir!r}"
)
def test_source_has_windows_branch_using_hermes_home(self):
root = Path(__file__).resolve().parents[2]
source = (root / "tools" / "environments" / "local.py").read_text(encoding="utf-8")
assert "if _IS_WINDOWS:" in source
assert "get_hermes_home" in source
assert 'cache_dir = get_hermes_home() / "cache" / "terminal"' in source
class TestLocalEnvironmentPathInjectionGated:
"""The /usr/bin PATH injection in _make_run_env must be POSIX-only."""
def test_source_gates_path_injection(self):
root = Path(__file__).resolve().parents[2]
source = (root / "tools" / "environments" / "local.py").read_text(encoding="utf-8")
# The fix wraps the injection in `if not _IS_WINDOWS`.
assert 'not _IS_WINDOWS and "/usr/bin" not in existing_path.split(":")' in source
# ---------------------------------------------------------------------------
# cli.py git path normalization
# ---------------------------------------------------------------------------
class TestGitBashPathNormalization:
"""_normalize_git_bash_path should turn /c/Users/... into C:\\Users\\...
on Windows and leave paths unchanged on POSIX."""
def test_posix_noop(self):
"""Must NOT mutate paths on Linux/macOS."""
from cli import _normalize_git_bash_path
if sys.platform != "win32":
assert _normalize_git_bash_path("/home/teknium/foo") == "/home/teknium/foo"
assert _normalize_git_bash_path("/c/Users/foo") == "/c/Users/foo"
assert _normalize_git_bash_path("C:/Users/foo") == "C:/Users/foo"
assert _normalize_git_bash_path(None) is None
def test_empty_string_preserved(self):
from cli import _normalize_git_bash_path
assert _normalize_git_bash_path("") == ""
def test_windows_translation(self, monkeypatch):
"""Simulate Windows and verify /c/Users/... becomes C:\\Users\\..."""
import cli as cli_mod
monkeypatch.setattr(cli_mod.sys, "platform", "win32")
assert cli_mod._normalize_git_bash_path("/c/Users/foo") == r"C:\Users\foo"
assert cli_mod._normalize_git_bash_path("/C/Users/foo") == r"C:\Users\foo"
assert cli_mod._normalize_git_bash_path("/cygdrive/d/data") == r"D:\data"
assert cli_mod._normalize_git_bash_path("/mnt/c/Users") == r"C:\Users"
# Already-native path is preserved
assert cli_mod._normalize_git_bash_path(r"C:\Users\foo") == r"C:\Users\foo"
# Forward-slash Windows path is preserved (git on Windows often
# returns this form; it's valid for both bash and Python, so we
# don't need to translate).
assert cli_mod._normalize_git_bash_path("C:/Users/foo") == "C:/Users/foo"
class TestWorktreeSymlinkFallback:
""".worktreeinclude directory symlinks must fall back to copytree on
Windows (where symlink creation requires admin / Dev Mode)."""
def test_source_has_symlink_fallback(self):
root = Path(__file__).resolve().parents[2]
source = (root / "cli.py").read_text(encoding="utf-8")
# Look for the try/except that handles OSError around os.symlink
# with a shutil.copytree fallback.
assert "os.symlink(str(src_resolved), str(dst))" in source
assert "except (OSError, NotImplementedError)" in source
assert "shutil.copytree" in source
assert 'sys.platform == "win32"' in source
# ---------------------------------------------------------------------------
# Gateway detached watcher — Windows creationflags
# ---------------------------------------------------------------------------
class TestGatewayDetachedWatcherWindowsFlags:
"""launch_detached_profile_gateway_restart and the in-gateway update
launcher must use CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS on
Windows, not silent start_new_session=True."""
def test_hermes_cli_gateway_uses_compat_kwargs(self):
root = Path(__file__).resolve().parents[2]
source = (root / "hermes_cli" / "gateway.py").read_text(encoding="utf-8")
assert "windows_detach_popen_kwargs" in source, (
"hermes_cli/gateway.py must use the platform-aware detach helper"
)
# The legacy start_new_session=True on the outer Popen should be
# replaced by **windows_detach_popen_kwargs(). Inside the watcher
# STRING the old pattern is replaced by explicit creationflags.
assert "**windows_detach_popen_kwargs()" in source
def test_gateway_run_update_has_windows_branch(self):
root = Path(__file__).resolve().parents[2]
source = (root / "gateway" / "run.py").read_text(encoding="utf-8")
# Both the /restart and /update paths must have sys.platform=='win32' branches.
assert 'if sys.platform == "win32":' in source
# Windows branch uses windows_detach_popen_kwargs
assert "windows_detach_popen_kwargs" in source
+34 -11
View File
@@ -708,7 +708,16 @@ def _run_chrome_fallback_command(
)
return {"success": False, "error": hint}
cmd_prefix = ["npx", "agent-browser"] if browser_cmd == "npx agent-browser" else [browser_cmd]
# On Windows npx is npx.cmd — use shutil.which so CreateProcessW can
# execute the batch shim. shutil.which honours PATHEXT on Windows and
# returns the plain executable on POSIX. If npx isn't on PATH (Termux,
# bare container), fall back to the bare name and let Popen raise with
# a readable "FileNotFoundError: 'npx'" rather than WinError 193.
if browser_cmd == "npx agent-browser":
_npx_bin = shutil.which("npx") or "npx"
cmd_prefix = [_npx_bin, "agent-browser"]
else:
cmd_prefix = [browser_cmd]
base_args = cmd_prefix + ["--engine", "chrome", "--session", tmp_session, "--json"]
task_socket_dir = os.path.join(_socket_safe_tmpdir(), f"agent-browser-{tmp_session}")
@@ -742,7 +751,7 @@ def _run_chrome_fallback_command(
proc.wait()
return {"success": False, "error": f"Chrome fallback '{cmd}' timed out"}
try:
with open(stdout_path, "r") as f:
with open(stdout_path, "r", encoding="utf-8") as f:
stdout = f.read().strip()
if stdout:
return json.loads(stdout.split("\n")[-1])
@@ -1101,7 +1110,7 @@ def _write_owner_pid(socket_dir: str, session_name: str) -> None:
"""
try:
path = os.path.join(socket_dir, f"{session_name}.owner_pid")
with open(path, "w") as f:
with open(path, "w", encoding="utf-8") as f:
f.write(str(os.getpid()))
except OSError as exc:
logger.debug("Could not write owner_pid file for %s: %s",
@@ -1165,7 +1174,7 @@ def _reap_orphaned_browser_sessions():
owner_alive: Optional[bool] = None # None = owner_pid missing/unreadable
if os.path.isfile(owner_pid_file):
try:
owner_pid = int(Path(owner_pid_file).read_text().strip())
owner_pid = int(Path(owner_pid_file).read_text(encoding="utf-8").strip())
try:
os.kill(owner_pid, 0)
owner_alive = True
@@ -1175,6 +1184,10 @@ def _reap_orphaned_browser_sessions():
# Owner exists but we can't signal it (different uid).
# Treat as alive — don't reap someone else's session.
owner_alive = True
except OSError:
# Windows: gone PID raises OSError (WinError 87) instead
# of ProcessLookupError. Treat as dead to match POSIX.
owner_alive = False
except (ValueError, OSError):
owner_alive = None # corrupt file — fall through
@@ -1196,7 +1209,7 @@ def _reap_orphaned_browser_sessions():
continue
try:
daemon_pid = int(Path(pid_file).read_text().strip())
daemon_pid = int(Path(pid_file).read_text(encoding="utf-8").strip())
except (ValueError, OSError):
shutil.rmtree(socket_dir, ignore_errors=True)
continue
@@ -1211,6 +1224,11 @@ def _reap_orphaned_browser_sessions():
except PermissionError:
# Alive but owned by someone else — leave it alone
continue
except OSError:
# Windows raises OSError (WinError 87) for a gone PID — treat
# as dead and clean up, mirroring the ProcessLookupError branch.
shutil.rmtree(socket_dir, ignore_errors=True)
continue
# Daemon is alive and its owner is dead (or legacy + untracked). Reap.
try:
@@ -1759,7 +1777,12 @@ def _run_browser_command(
# Keep concrete executable paths intact, even when they contain spaces.
# Only the synthetic npx fallback needs to expand into multiple argv items.
cmd_prefix = ["npx", "agent-browser"] if browser_cmd == "npx agent-browser" else [browser_cmd]
# shutil.which resolves npx → npx.cmd on Windows; bare "npx" stays on POSIX.
if browser_cmd == "npx agent-browser":
_npx_bin = shutil.which("npx") or "npx"
cmd_prefix = [_npx_bin, "agent-browser"]
else:
cmd_prefix = [browser_cmd]
cmd_parts = cmd_prefix + backend_args + [
"--json",
@@ -1811,7 +1834,7 @@ def _run_browser_command(
# Detect AppArmor user namespace restrictions (Ubuntu 23.10+)
_userns_restrict = "/proc/sys/kernel/apparmor_restrict_unprivileged_userns"
try:
with open(_userns_restrict) as _f:
with open(_userns_restrict, encoding="utf-8") as _f:
if _f.read().strip() == "1":
_needs_sandbox_bypass = True
logger.debug(
@@ -1856,9 +1879,9 @@ def _run_browser_command(
result = {"success": False, "error": f"Command timed out after {timeout} seconds"}
# Fall through to fallback check below
else:
with open(stdout_path, "r") as f:
with open(stdout_path, "r", encoding="utf-8") as f:
stdout = f.read()
with open(stderr_path, "r") as f:
with open(stderr_path, "r", encoding="utf-8") as f:
stderr = f.read()
returncode = proc.returncode
@@ -3157,7 +3180,7 @@ def _cleanup_single_browser_session(task_id: str) -> None:
pid_file = os.path.join(socket_dir, f"{session_name}.pid")
if os.path.isfile(pid_file):
try:
daemon_pid = int(Path(pid_file).read_text().strip())
daemon_pid = int(Path(pid_file).read_text(encoding="utf-8").strip())
os.kill(daemon_pid, signal.SIGTERM)
logger.debug("Killed daemon pid %s for %s", daemon_pid, session_name)
except (ProcessLookupError, ValueError, PermissionError, OSError):
@@ -3300,7 +3323,7 @@ def _running_in_docker() -> bool:
if os.path.exists("/.dockerenv"):
return True
try:
with open("/proc/1/cgroup", "rt") as fp:
with open("/proc/1/cgroup", "rt", encoding="utf-8") as fp:
return "docker" in fp.read()
except OSError:
return False
+185 -43
View File
@@ -47,10 +47,13 @@ import uuid
_IS_WINDOWS = platform.system() == "Windows"
from typing import Any, Dict, List, Optional
# Availability gate: UDS requires a POSIX OS
# Availability gate. On Windows we fall back to loopback TCP for the
# sandbox RPC transport (AF_UNIX is unreliable on Windows Python) — see
# ``_use_tcp_rpc`` in ``_execute_local`` below. That makes execute_code
# available on every platform Hermes itself runs on.
logger = logging.getLogger(__name__)
SANDBOX_AVAILABLE = sys.platform != "win32"
SANDBOX_AVAILABLE = True
# The 7 tools allowed inside the sandbox. The intersection of this list
# and the session's enabled tools determines which stubs are generated.
@@ -70,6 +73,85 @@ DEFAULT_MAX_TOOL_CALLS = 50
MAX_STDOUT_BYTES = 50_000 # 50 KB
MAX_STDERR_BYTES = 10_000 # 10 KB
# Environment variable scrubbing rules (shared between the local + remote
# backends). Secret-substring block is applied first; anything left must
# match either a safe prefix or, on Windows, an OS-essential name.
_SAFE_ENV_PREFIXES = ("PATH", "HOME", "USER", "LANG", "LC_", "TERM",
"TMPDIR", "TMP", "TEMP", "SHELL", "LOGNAME",
"XDG_", "PYTHONPATH", "VIRTUAL_ENV", "CONDA",
"HERMES_")
_SECRET_SUBSTRINGS = ("KEY", "TOKEN", "SECRET", "PASSWORD", "CREDENTIAL",
"PASSWD", "AUTH")
# Windows-only: a handful of variables are required by the OS/CRT itself.
# Without them, even stdlib calls like ``socket.socket()`` fail with
# WinError 10106 (Winsock can't locate mswsock.dll) and ``subprocess``
# can't resolve cmd.exe. These are well-known OS paths, not secrets, so
# we allow them through by exact name. The _SECRET_SUBSTRINGS block
# still runs as a safety net (none of these names match those substrings).
_WINDOWS_ESSENTIAL_ENV_VARS = frozenset({
"SYSTEMROOT", # %SYSTEMROOT%\System32 — Winsock needs this
"SYSTEMDRIVE", # C: (or wherever Windows lives)
"WINDIR", # usually same as SYSTEMROOT
"COMSPEC", # cmd.exe path — subprocess shell=True needs it
"PATHEXT", # .COM;.EXE;.BAT;... — shell lookup
"OS", # "Windows_NT" — some tools gate on this
"PROCESSOR_ARCHITECTURE",
"NUMBER_OF_PROCESSORS",
"PUBLIC", # C:\Users\Public
"ALLUSERSPROFILE", # C:\ProgramData — some stdlib paths use it
"PROGRAMDATA", # C:\ProgramData
"PROGRAMFILES",
"PROGRAMFILES(X86)",
"PROGRAMW6432",
"APPDATA", # %USERPROFILE%\AppData\Roaming — Python uses it
"LOCALAPPDATA", # %USERPROFILE%\AppData\Local
"USERPROFILE", # C:\Users\<name> — Python's expanduser uses it
"USERDOMAIN",
"USERNAME",
"HOMEDRIVE", # C:
"HOMEPATH", # \Users\<name>
"COMPUTERNAME",
})
def _scrub_child_env(source_env, is_passthrough=None, is_windows=None):
"""Produce the scrubbed child-process env for execute_code.
Rules (order matters):
1. Passthrough vars (skill- or config-declared) always pass.
2. Secret-substring names (KEY/TOKEN/etc.) are blocked.
3. Names matching a safe prefix pass.
4. On Windows, a small OS-essential allowlist passes by exact name
without these the child can't even create a socket or spawn a
subprocess.
Extracted into a helper so tests can exercise the logic without
spawning a subprocess.
"""
if is_passthrough is None:
try:
from tools.env_passthrough import is_env_passthrough as _ep
except Exception:
_ep = lambda _: False # noqa: E731
is_passthrough = _ep
if is_windows is None:
is_windows = _IS_WINDOWS
scrubbed = {}
for k, v in source_env.items():
if is_passthrough(k):
scrubbed[k] = v
continue
if any(s in k.upper() for s in _SECRET_SUBSTRINGS):
continue
if any(k.startswith(p) for p in _SAFE_ENV_PREFIXES):
scrubbed[k] = v
continue
if is_windows and k.upper() in _WINDOWS_ESSENTIAL_ENV_VARS:
scrubbed[k] = v
return scrubbed
def check_sandbox_requirements() -> bool:
"""Code execution sandbox requires a POSIX OS for Unix domain sockets."""
@@ -235,10 +317,27 @@ _call_lock = threading.Lock()
''' + _COMMON_HELPERS + '''\
def _connect():
"""Connect to the parent's RPC server via the transport it picked.
HERMES_RPC_SOCKET can be either:
- a filesystem path (POSIX Unix domain socket the default on
Linux and macOS)
- a string of the form ``tcp://127.0.0.1:<port>`` (Windows, where
AF_UNIX is unreliable the parent falls back to loopback TCP)
"""
global _sock
if _sock is None:
_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
_sock.connect(os.environ["HERMES_RPC_SOCKET"])
endpoint = os.environ["HERMES_RPC_SOCKET"]
if endpoint.startswith("tcp://"):
# tcp://host:port (host is always 127.0.0.1 in practice — we
# only bind loopback server-side)
_host_port = endpoint[len("tcp://"):]
_host, _, _port = _host_port.rpartition(":")
_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
_sock.connect((_host or "127.0.0.1", int(_port)))
else:
_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
_sock.connect(endpoint)
_sock.settimeout(300)
return _sock
@@ -291,9 +390,12 @@ def _call(tool_name, args):
req_file = os.path.join(_RPC_DIR, f"req_{seq_str}")
res_file = os.path.join(_RPC_DIR, f"res_{seq_str}")
# Write request atomically (write to .tmp, then rename)
# Write request atomically (write to .tmp, then rename).
# encoding="utf-8" is critical: on Windows-hosted remote backends
# (or any non-UTF-8 locale) the default open() mode would mangle
# non-ASCII chars in tool args when encoding them as JSON.
tmp = req_file + ".tmp"
with open(tmp, "w") as f:
with open(tmp, "w", encoding="utf-8") as f:
json.dump({"tool": tool_name, "args": args, "seq": seq}, f)
os.rename(tmp, req_file)
@@ -306,7 +408,7 @@ def _call(tool_name, args):
time.sleep(poll_interval)
poll_interval = min(poll_interval * 1.2, 0.25) # Back off to 250ms
with open(res_file) as f:
with open(res_file, encoding="utf-8") as f:
raw = f.read()
# Clean up response file
@@ -415,7 +517,7 @@ def _rpc_server_loop(
# their status prints don't leak into the CLI spinner.
try:
_real_stdout, _real_stderr = sys.stdout, sys.stderr
devnull = open(os.devnull, "w")
devnull = open(os.devnull, "w", encoding="utf-8")
try:
sys.stdout = devnull
sys.stderr = devnull
@@ -689,7 +791,7 @@ def _rpc_poll_loop(
# Dispatch through the standard tool handler
try:
_real_stdout, _real_stderr = sys.stdout, sys.stderr
devnull = open(os.devnull, "w")
devnull = open(os.devnull, "w", encoding="utf-8")
try:
sys.stdout = devnull
sys.stderr = devnull
@@ -954,7 +1056,8 @@ def execute_code(
"""
if not SANDBOX_AVAILABLE:
return json.dumps({
"error": "execute_code is not available on Windows. Use normal tool calls instead."
"error": "execute_code sandbox is unavailable in this environment. "
"Use normal tool calls (terminal, read_file, write_file, ...) instead."
})
if not code or not code.strip():
@@ -988,8 +1091,22 @@ def execute_code(
# Use /tmp on macOS to avoid the long /var/folders/... path that pushes
# Unix domain socket paths past the 104-byte macOS AF_UNIX limit.
# On Linux, tempfile.gettempdir() already returns /tmp.
#
# Windows: Python 3.9+ added partial AF_UNIX support but the file-backed
# variant is flaky across Windows builds (requires Windows 10 1803+,
# still fails under some configurations, and the socket file can't live
# on the same temp drive as the script). Fall back to loopback TCP —
# same ephemeral port, same 1-connection listen queue, same serialized
# request/response framing. The generated client reads the transport
# selector from HERMES_RPC_SOCKET (path vs. ``tcp://host:port``).
_sock_tmpdir = "/tmp" if sys.platform == "darwin" else tempfile.gettempdir()
sock_path = os.path.join(_sock_tmpdir, f"hermes_rpc_{uuid.uuid4().hex}.sock")
_use_tcp_rpc = _IS_WINDOWS
if _use_tcp_rpc:
sock_path = None # not used on Windows; TCP endpoint stored below
rpc_endpoint = None # set after bind()
else:
sock_path = os.path.join(_sock_tmpdir, f"hermes_rpc_{uuid.uuid4().hex}.sock")
rpc_endpoint = sock_path
tool_call_log: list = []
tool_call_counter = [0] # mutable so the RPC thread can increment
@@ -997,21 +1114,42 @@ def execute_code(
server_sock = None
try:
# Write the auto-generated hermes_tools module
# Write the auto-generated hermes_tools module.
# encoding="utf-8" is required on Windows — the stub and user code
# both contain non-ASCII characters (em-dashes in docstrings, plus
# whatever the user script carries). Python's default open() uses
# the system locale on Windows (cp1252 typically), which corrupts
# those bytes; the child then fails to import with a SyntaxError
# ("'utf-8' codec can't decode byte 0x97 in position ...") because
# Python source files are decoded as UTF-8 by default (PEP 3120).
# sandbox_tools is already the correct set (intersection with session
# tools, or SANDBOX_ALLOWED_TOOLS as fallback — see lines above).
tools_src = generate_hermes_tools_module(list(sandbox_tools))
with open(os.path.join(tmpdir, "hermes_tools.py"), "w") as f:
with open(os.path.join(tmpdir, "hermes_tools.py"), "w", encoding="utf-8") as f:
f.write(tools_src)
# Write the user's script
with open(os.path.join(tmpdir, "script.py"), "w") as f:
with open(os.path.join(tmpdir, "script.py"), "w", encoding="utf-8") as f:
f.write(code)
# --- Start UDS server ---
server_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
server_sock.bind(sock_path)
os.chmod(sock_path, 0o600)
# --- Start RPC server ---
# Two transports:
# POSIX: AF_UNIX stream socket on sock_path, chmod 0600 for
# owner-only access. Filesystem permissions gate the socket.
# Windows: AF_INET stream socket on 127.0.0.1 with an ephemeral
# port. No filesystem permission story, but loopback-only bind
# means only the current user's processes (not remote) can
# connect. HERMES_RPC_SOCKET is set to ``tcp://127.0.0.1:<port>``
# which the generated client parses to pick AF_INET.
if _use_tcp_rpc:
server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_sock.bind(("127.0.0.1", 0)) # ephemeral port
_host, _port = server_sock.getsockname()[:2]
rpc_endpoint = f"tcp://{_host}:{_port}"
else:
server_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
server_sock.bind(sock_path)
os.chmod(sock_path, 0o600)
server_sock.listen(1)
rpc_thread = threading.Thread(
@@ -1030,31 +1168,32 @@ def execute_code(
# generated scripts. The child accesses tools via RPC, not direct API.
# Exception: env vars declared by loaded skills (via env_passthrough
# registry) or explicitly allowed by the user in config.yaml
# (terminal.env_passthrough) are passed through.
_SAFE_ENV_PREFIXES = ("PATH", "HOME", "USER", "LANG", "LC_", "TERM",
"TMPDIR", "TMP", "TEMP", "SHELL", "LOGNAME",
"XDG_", "PYTHONPATH", "VIRTUAL_ENV", "CONDA",
"HERMES_")
_SECRET_SUBSTRINGS = ("KEY", "TOKEN", "SECRET", "PASSWORD", "CREDENTIAL",
"PASSWD", "AUTH")
try:
from tools.env_passthrough import is_env_passthrough as _is_passthrough
except Exception:
_is_passthrough = lambda _: False # noqa: E731
child_env = {}
for k, v in os.environ.items():
# Passthrough vars (skill-declared or user-configured) always pass.
if _is_passthrough(k):
child_env[k] = v
continue
# Block vars with secret-like names.
if any(s in k.upper() for s in _SECRET_SUBSTRINGS):
continue
# Allow vars with known safe prefixes.
if any(k.startswith(p) for p in _SAFE_ENV_PREFIXES):
child_env[k] = v
child_env["HERMES_RPC_SOCKET"] = sock_path
# (terminal.env_passthrough) are passed through. On Windows, a small
# OS-essential allowlist (SYSTEMROOT, WINDIR, COMSPEC, ...) is also
# passed through — without those, the child can't create a socket
# or spawn a subprocess. See ``_scrub_child_env`` for the rules.
child_env = _scrub_child_env(os.environ)
child_env["HERMES_RPC_SOCKET"] = rpc_endpoint
child_env["PYTHONDONTWRITEBYTECODE"] = "1"
# Force UTF-8 for the child's stdio and default file encoding.
#
# Without this, on Windows sys.stdout is bound to the console code
# page (cp1252 on US-locale installs), and any script that does
# ``print("café")`` or ``print("→")`` crashes with:
#
# UnicodeEncodeError: 'charmap' codec can't encode character
# '\u2192' in position N: character maps to <undefined>
#
# PYTHONIOENCODING fixes sys.stdin/stdout/stderr.
# PYTHONUTF8=1 enables "UTF-8 mode" (PEP 540) which additionally
# makes ``open()``'s default encoding UTF-8, so user scripts that
# write files without specifying encoding= also work correctly.
#
# On POSIX both values usually match the locale default already,
# so setting them is harmless belt-and-suspenders for environments
# with a C/POSIX locale (containers, minimal base images).
child_env["PYTHONIOENCODING"] = "utf-8"
child_env["PYTHONUTF8"] = "1"
# Ensure the hermes-agent root is importable in the sandbox so
# repo-root modules are available to child scripts. We also prepend
# the staging tmpdir so ``from hermes_tools import ...`` resolves even
@@ -1302,7 +1441,10 @@ def execute_code(
import shutil
shutil.rmtree(tmpdir, ignore_errors=True)
try:
os.unlink(sock_path)
# Only UDS has a filesystem socket to unlink; TCP sockets are
# freed by server_sock.close() above.
if sock_path:
os.unlink(sock_path)
except OSError:
pass # already cleaned up or never created
+52 -15
View File
@@ -99,12 +99,33 @@ def get_sandbox_dir() -> Path:
def _pipe_stdin(proc: subprocess.Popen, data: str) -> None:
"""Write *data* to proc.stdin on a daemon thread to avoid pipe-buffer deadlocks."""
"""Write *data* to proc.stdin on a daemon thread to avoid pipe-buffer deadlocks.
On Windows, text-mode stdin (``text=True`` / ``encoding="utf-8"``)
translates ``\\n`` ``\\r\\n`` as the data flows through the pipe
which corrupts every write_file / patch call because the bytes that
land on disk include injected carriage returns. The file IS created,
but every subsequent byte-count / content compare against the
caller's ``\\n``-only string fails.
Workaround: write through ``proc.stdin.buffer`` (the underlying byte
buffer), encoding to UTF-8 ourselves. That bypasses Python's
newline translation entirely on every platform. No behaviour change
on POSIX the byte sequence is identical to what text-mode would
produce there.
"""
def _write():
try:
proc.stdin.write(data)
proc.stdin.close()
# proc.stdin is a TextIOWrapper when text=True was set on the
# Popen. Its ``.buffer`` attribute is the raw BufferedWriter
# that bypasses newline translation. When Popen was created
# in byte mode, proc.stdin is already a BufferedWriter with
# no ``.buffer`` attribute — fall back to .write() directly.
raw = data.encode("utf-8") if isinstance(data, str) else data
target = getattr(proc.stdin, "buffer", proc.stdin)
target.write(raw)
target.close()
except (BrokenPipeError, OSError):
pass
@@ -137,7 +158,7 @@ def _load_json_store(path: Path) -> dict:
"""Load a JSON file as a dict, returning ``{}`` on any error."""
if path.exists():
try:
return json.loads(path.read_text())
return json.loads(path.read_text(encoding="utf-8"))
except Exception:
pass
return {}
@@ -146,7 +167,7 @@ def _load_json_store(path: Path) -> dict:
def _save_json_store(path: Path, data: dict) -> None:
"""Write *data* as pretty-printed JSON to *path*."""
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, indent=2))
path.write_text(json.dumps(data, indent=2), encoding="utf-8")
def _file_mtime_key(host_path: str) -> tuple[float, int] | None:
@@ -339,15 +360,24 @@ class BaseEnvironment(ABC):
# change the working directory (e.g. bashrc `cd ~`). Without this,
# pwd -P captures the profile's directory, not terminal.cwd.
_quoted_cwd = shlex.quote(self.cwd)
# Quote the snapshot / cwd-file paths so Git Bash on Windows handles
# ``C:/Users/...``-shaped paths without glob-splitting the colon or
# tripping on drive letters. On POSIX this is a no-op (no colons /
# special chars in a /tmp path). Previously unquoted interpolation
# caused ``C:/Users/.../hermes-snap-*.sh: No such file or directory``
# errors on Windows, leaking via stderr (merged into stdout on Linux
# backends) into every terminal-tool response.
_quoted_snap = shlex.quote(self._snapshot_path)
_quoted_cwd_file = shlex.quote(self._cwd_file)
bootstrap = (
f"export -p > {self._snapshot_path}\n"
f"declare -f | grep -vE '^_[^_]' >> {self._snapshot_path}\n"
f"alias -p >> {self._snapshot_path}\n"
f"echo 'shopt -s expand_aliases' >> {self._snapshot_path}\n"
f"echo 'set +e' >> {self._snapshot_path}\n"
f"echo 'set +u' >> {self._snapshot_path}\n"
f"export -p > {_quoted_snap}\n"
f"declare -f | grep -vE '^_[^_]' >> {_quoted_snap}\n"
f"alias -p >> {_quoted_snap}\n"
f"echo 'shopt -s expand_aliases' >> {_quoted_snap}\n"
f"echo 'set +e' >> {_quoted_snap}\n"
f"echo 'set +u' >> {_quoted_snap}\n"
f"builtin cd {_quoted_cwd} 2>/dev/null || true\n"
f"pwd -P > {self._cwd_file} 2>/dev/null || true\n"
f"pwd -P > {_quoted_cwd_file} 2>/dev/null || true\n"
f"printf '\\n{self._cwd_marker}%s{self._cwd_marker}\\n' \"$(pwd -P)\"\n"
)
try:
@@ -389,6 +419,13 @@ class BaseEnvironment(ABC):
re-dumps env vars, and emits CWD markers."""
escaped = command.replace("'", "'\\''")
# Quote the snapshot / cwd-file paths so Git Bash on Windows handles
# ``C:/Users/...``-shaped paths without glob-splitting the colon or
# tripping on drive letters. POSIX paths are unaffected. See
# :meth:`init_session` for the same fix on the bootstrap block.
_quoted_snap = shlex.quote(self._snapshot_path)
_quoted_cwd_file = shlex.quote(self._cwd_file)
parts = []
# Source snapshot (env vars from previous commands).
@@ -399,7 +436,7 @@ class BaseEnvironment(ABC):
# silent here, but the redirect is harmless.
if self._snapshot_ready:
parts.append(
f"source {self._snapshot_path} >/dev/null 2>&1 || true"
f"source {_quoted_snap} >/dev/null 2>&1 || true"
)
# Preserve bare ``~`` expansion, but rewrite ``~/...`` through
@@ -414,10 +451,10 @@ class BaseEnvironment(ABC):
# Re-dump env vars to snapshot (last-writer-wins for concurrent calls)
if self._snapshot_ready:
parts.append(f"export -p > {self._snapshot_path} 2>/dev/null || true")
parts.append(f"export -p > {_quoted_snap} 2>/dev/null || true")
# Write CWD to file (local reads this) and stdout marker (remote parses this)
parts.append(f"pwd -P > {self._cwd_file} 2>/dev/null || true")
parts.append(f"pwd -P > {_quoted_cwd_file} 2>/dev/null || true")
# Use a distinct line for the marker. The leading \n ensures
# the marker starts on its own line even if the command doesn't
# end with a newline (e.g. printf 'exact'). We'll strip this
+1 -1
View File
@@ -284,7 +284,7 @@ class FileSyncManager:
# Windows: no flock — run without serialization
self._sync_back_impl()
return
lock_fd = open(lock_path, "w")
lock_fd = open(lock_path, "w", encoding="utf-8")
try:
fcntl.flock(lock_fd, fcntl.LOCK_EX)
self._sync_back_impl()
+53 -3
View File
@@ -9,6 +9,7 @@ import signal
import subprocess
import tempfile
import time
from pathlib import Path
from tools.environments.base import BaseEnvironment, _pipe_stdin
@@ -189,6 +190,25 @@ def _find_bash() -> str:
if custom and os.path.isfile(custom):
return custom
# Prefer our own portable Git install first — this way a broken or
# partially-uninstalled system Git can't hijack the bash lookup. The
# install.ps1 installer always drops portable Git here when the user
# didn't already have a working system Git.
#
# Layouts (both checked so upgrades between MinGit and PortableGit
# installs work transparently):
# PortableGit: %LOCALAPPDATA%\hermes\git\bin\bash.exe (primary)
# MinGit: %LOCALAPPDATA%\hermes\git\usr\bin\bash.exe (legacy/32-bit fallback)
_local_appdata = os.environ.get("LOCALAPPDATA", "")
_hermes_portable_git = os.path.join(_local_appdata, "hermes", "git") if _local_appdata else ""
if _hermes_portable_git:
for candidate in (
os.path.join(_hermes_portable_git, "bin", "bash.exe"), # PortableGit (primary)
os.path.join(_hermes_portable_git, "usr", "bin", "bash.exe"), # MinGit fallback
):
if os.path.isfile(candidate):
return candidate
found = shutil.which("bash")
if found:
return found
@@ -196,7 +216,7 @@ def _find_bash() -> str:
for candidate in (
os.path.join(os.environ.get("ProgramFiles", r"C:\Program Files"), "Git", "bin", "bash.exe"),
os.path.join(os.environ.get("ProgramFiles(x86)", r"C:\Program Files (x86)"), "Git", "bin", "bash.exe"),
os.path.join(os.environ.get("LOCALAPPDATA", ""), "Programs", "Git", "bin", "bash.exe"),
os.path.join(_local_appdata, "Programs", "Git", "bin", "bash.exe"),
):
if candidate and os.path.isfile(candidate):
return candidate
@@ -235,7 +255,15 @@ def _make_run_env(env: dict) -> dict:
elif k not in _HERMES_PROVIDER_ENV_BLOCKLIST or _is_passthrough(k):
run_env[k] = v
existing_path = run_env.get("PATH", "")
if "/usr/bin" not in existing_path.split(":"):
# The "/usr/bin not already present → inject sane POSIX path" heuristic
# only makes sense on POSIX. On Windows the PATH separator is ";"
# (the split(":") above turns a full Windows PATH into a single
# unrecognisable chunk, which then triggers prepending POSIX paths
# to a Windows PATH — completely wrong). Skip the injection entirely
# on Windows; the native PATH already points at whatever shell
# Hermes is driving via _find_bash (Git Bash), and Git Bash itself
# prepends its MSYS2 /usr/bin equivalent via the shell-init files.
if not _IS_WINDOWS and "/usr/bin" not in existing_path.split(":"):
run_env["PATH"] = f"{existing_path}:{_SANE_PATH}" if existing_path else _SANE_PATH
# Per-profile HOME isolation: redirect system tool configs (git, ssh, gh,
@@ -357,7 +385,29 @@ class LocalEnvironment(BaseEnvironment):
Check the environment configured for this backend first so callers can
override the temp root explicitly (for example via terminal.env or a
custom TMPDIR), then fall back to the host process environment.
**Windows:** hardcoded ``/tmp`` is wrong in two ways native Python
can't open the path, and the Windows default temp (``%TEMP%``) often
contains spaces (``C:\\Users\\Some Name\\AppData\\Local\\Temp``) that
break unquoted bash interpolations. Use a dedicated cache dir under
``HERMES_HOME`` instead single-word path, guaranteed to exist, same
string resolves in both Git Bash and native Python.
"""
if _IS_WINDOWS:
# Derive a Windows-safe temp dir under HERMES_HOME. Using
# forward slashes makes the same string work unchanged in bash
# command interpolations AND in Python ``open()`` — Windows
# accepts forward slashes in filesystem paths, and we control
# the path so we can guarantee no spaces.
try:
from hermes_constants import get_hermes_home
cache_dir = get_hermes_home() / "cache" / "terminal"
except Exception:
cache_dir = Path(tempfile.gettempdir()) / "hermes_terminal"
cache_dir.mkdir(parents=True, exist_ok=True)
# Force forward slashes so the same string serves both contexts.
return str(cache_dir).replace("\\", "/")
for env_var in ("TMPDIR", "TMP", "TEMP"):
candidate = self.env.get(env_var) or os.environ.get(env_var)
if candidate and candidate.startswith("/"):
@@ -512,7 +562,7 @@ class LocalEnvironment(BaseEnvironment):
``_run_bash`` recovery path will resolve a safe fallback if needed.
"""
try:
with open(self._cwd_file) as f:
with open(self._cwd_file, encoding="utf-8") as f:
cwd_path = f.read().strip()
if cwd_path and os.path.isdir(cwd_path):
self.cwd = cwd_path
+12 -2
View File
@@ -966,11 +966,21 @@ class ShellFileOperations(FileOperations):
verify_result = self._exec(verify_cmd)
if verify_result.exit_code != 0:
return PatchResult(error=f"Post-write verification failed: could not re-read {path}")
if verify_result.stdout != new_content:
# Normalize line endings before comparing. On Windows, Python's
# default text-mode ``open()`` translates ``\n`` → ``\r\n`` on
# write, so the file on disk legitimately holds CRLFs while our
# ``new_content`` string has bare LFs. Without this normalization
# every patch on Windows returns a bogus "wrote 39, read 42"
# false-negative even though the edit landed correctly. POSIX
# backends don't translate, so this is a no-op there.
_verify_stdout_normalized = verify_result.stdout.replace("\r\n", "\n").replace("\r", "\n")
_new_content_normalized = new_content.replace("\r\n", "\n").replace("\r", "\n")
if _verify_stdout_normalized != _new_content_normalized:
return PatchResult(error=(
f"Post-write verification failed for {path}: on-disk content "
f"differs from intended write "
f"(wrote {len(new_content)} chars, read back {len(verify_result.stdout)}). "
f"(wrote {len(_new_content_normalized)} chars, read back "
f"{len(_verify_stdout_normalized)} chars after normalizing line endings). "
"The patch did not persist. Re-read the file and try again."
))
+1 -1
View File
@@ -1992,7 +1992,7 @@ def _snapshot_child_pids() -> set:
# Linux: read from /proc
try:
children_path = f"/proc/{my_pid}/task/{my_pid}/children"
with open(children_path) as f:
with open(children_path, encoding="utf-8") as f:
return {int(p) for p in f.read().split() if p.strip()}
except (FileNotFoundError, OSError, ValueError):
pass
+5 -1
View File
@@ -407,7 +407,11 @@ class ProcessRegistry:
try:
os.kill(pid, 0)
return True
except (ProcessLookupError, PermissionError):
except (ProcessLookupError, PermissionError, OSError):
# OSError covers Windows' WinError 87 for a gone PID, and the
# ``WinError 5 Access denied`` case — treat both as "can't probe
# or process is gone", which matches the conservative
# "not alive" semantics callers already handle.
return False
def _refresh_detached_session(self, session: Optional[ProcessSession]) -> Optional[ProcessSession]:
+7 -7
View File
@@ -169,7 +169,7 @@ def _scan_environments() -> List[EnvironmentInfo]:
continue
try:
with open(py_file, "r") as f:
with open(py_file, "r", encoding="utf-8") as f:
tree = ast.parse(f.read())
for node in ast.walk(tree):
@@ -333,7 +333,7 @@ async def _spawn_training_run(run_state: RunState, config_path: Path):
# File must stay open while the subprocess runs; we store the handle
# on run_state so _stop_training_run() can close it when done.
api_log_file = open(api_log, "w") # closed by _stop_training_run
api_log_file = open(api_log, "w", encoding="utf-8") # closed by _stop_training_run
run_state.api_log_file = api_log_file
run_state.api_process = subprocess.Popen(
["run-api"],
@@ -356,7 +356,7 @@ async def _spawn_training_run(run_state: RunState, config_path: Path):
# Step 2: Start the Tinker trainer
logger.info("[%s] Starting Tinker trainer: launch_training.py --config %s", run_id, config_path)
trainer_log_file = open(trainer_log, "w") # closed by _stop_training_run
trainer_log_file = open(trainer_log, "w", encoding="utf-8") # closed by _stop_training_run
run_state.trainer_log_file = trainer_log_file
run_state.trainer_process = subprocess.Popen(
[sys.executable, "launch_training.py", "--config", str(config_path)],
@@ -397,7 +397,7 @@ async def _spawn_training_run(run_state: RunState, config_path: Path):
logger.info("[%s] Starting environment: %s serve", run_id, env_info.file_path)
env_log_file = open(env_log, "w") # closed by _stop_training_run
env_log_file = open(env_log, "w", encoding="utf-8") # closed by _stop_training_run
run_state.env_log_file = env_log_file
run_state.env_process = subprocess.Popen(
[sys.executable, str(env_info.file_path), "serve", "--config", str(config_path)],
@@ -777,7 +777,7 @@ async def rl_start_training() -> str:
if "wandb_name" in _current_config and _current_config["wandb_name"]:
run_config["env"]["wandb_name"] = _current_config["wandb_name"]
with open(config_path, "w") as f:
with open(config_path, "w", encoding="utf-8") as f:
yaml.dump(run_config, f, default_flow_style=False)
# Create run state
@@ -1206,7 +1206,7 @@ async def rl_test_inference(
stderr_text = "\n".join(stderr_lines)
# Write logs to files for inspection outside CLI
with open(log_file, "w") as f:
with open(log_file, "w", encoding="utf-8") as f:
f.write(f"Command: {cmd_display}\n")
f.write(f"Working dir: {TINKER_ATROPOS_ROOT}\n")
f.write(f"Return code: {process.returncode}\n")
@@ -1238,7 +1238,7 @@ async def rl_test_inference(
# Parse the output JSONL file
if output_file.exists():
# Read JSONL file (one JSON object per line = one step)
with open(output_file, "r") as f:
with open(output_file, "r", encoding="utf-8") as f:
for line in f:
line = line.strip()
if not line:
+2 -2
View File
@@ -219,7 +219,7 @@ class GitHubAuth:
key_file = Path(key_path)
if not key_file.exists():
return None
private_key = key_file.read_text()
private_key = key_file.read_text(encoding="utf-8")
now = int(time.time())
payload = {
@@ -2667,7 +2667,7 @@ def append_audit_log(action: str, skill_name: str, source: str,
parts.append(extra)
line = " ".join(parts) + "\n"
try:
with open(AUDIT_LOG, "a") as f:
with open(AUDIT_LOG, "a", encoding="utf-8") as f:
f.write(line)
except OSError as e:
logger.debug("Could not write audit log: %s", e)
+3 -3
View File
@@ -126,7 +126,7 @@ def _read_failure_reason() -> str | None:
mtime = os.path.getmtime(p)
if (time.time() - mtime) >= _MARKER_TTL:
return None
with open(p, "r") as f:
with open(p, "r", encoding="utf-8") as f:
return f.read().strip()
except OSError:
return None
@@ -160,7 +160,7 @@ def _mark_install_failed(reason: str = ""):
try:
p = _failure_marker_path()
os.makedirs(os.path.dirname(p), exist_ok=True)
with open(p, "w") as f:
with open(p, "w", encoding="utf-8") as f:
f.write(reason)
except OSError:
pass
@@ -257,7 +257,7 @@ def _verify_cosign(checksums_path: str, sig_path: str, cert_path: str) -> bool |
def _verify_checksum(archive_path: str, checksums_path: str, archive_name: str) -> bool:
"""Verify SHA-256 of the archive against checksums.txt."""
expected = None
with open(checksums_path) as f:
with open(checksums_path, encoding="utf-8") as f:
for line in f:
# Format: "<hash> <filename>"
parts = line.strip().split(" ", 1)
+1 -1
View File
@@ -110,7 +110,7 @@ def detect_audio_environment() -> dict:
# WSL detection — PulseAudio bridge makes audio work in WSL.
# Only block if PULSE_SERVER is not configured.
try:
with open('/proc/version', 'r') as f:
with open('/proc/version', 'r', encoding="utf-8") as f:
if 'microsoft' in f.read().lower():
if os.environ.get('PULSE_SERVER'):
notices.append("Running in WSL with PulseAudio bridge")
+2 -2
View File
@@ -125,7 +125,7 @@ class CompressionConfig:
@classmethod
def from_yaml(cls, yaml_path: str) -> "CompressionConfig":
"""Load configuration from YAML file."""
with open(yaml_path, 'r') as f:
with open(yaml_path, 'r', encoding="utf-8") as f:
data = yaml.safe_load(f)
config = cls()
@@ -1174,7 +1174,7 @@ Write only the summary, starting with "[CONTEXT SUMMARY]:" prefix."""
# Save metrics
if self.config.metrics_enabled:
metrics_path = output_dir / self.config.metrics_output_file
with open(metrics_path, 'w') as f:
with open(metrics_path, 'w', encoding="utf-8") as f:
json.dump(self.aggregate_metrics.to_dict(), f, indent=2)
console.print(f"\n💾 Metrics saved to {metrics_path}")
+25 -9
View File
@@ -81,11 +81,14 @@ def _log_signal(signum: int, frame) -> None:
thread, and fall back to ``os._exit(0)`` so a wedged write/flush
can never strand the process.
"""
name = {
signal.SIGPIPE: "SIGPIPE",
signal.SIGTERM: "SIGTERM",
signal.SIGHUP: "SIGHUP",
}.get(signum, f"signal {signum}")
# SIGPIPE and SIGHUP don't exist on Windows — build the lookup
# dict from attributes that actually exist on the current platform.
_signal_names: dict[int, str] = {}
for _attr in ("SIGPIPE", "SIGTERM", "SIGHUP", "SIGINT", "SIGBREAK"):
_sig = getattr(signal, _attr, None)
if _sig is not None:
_signal_names[int(_sig)] = _attr
name = _signal_names.get(signum, f"signal {signum}")
try:
os.makedirs(os.path.dirname(_CRASH_LOG), exist_ok=True)
with open(_CRASH_LOG, "a", encoding="utf-8") as f:
@@ -140,10 +143,23 @@ def _log_signal(signum: int, frame) -> None:
# sys.exit(0) + _log_exit), which keeps the gateway alive as long as
# the main command pipe is still readable. Terminal signals still
# route through _log_signal so kills and hangups are diagnosable.
signal.signal(signal.SIGPIPE, signal.SIG_IGN)
signal.signal(signal.SIGTERM, _log_signal)
signal.signal(signal.SIGHUP, _log_signal)
signal.signal(signal.SIGINT, signal.SIG_IGN)
#
# SIGPIPE and SIGHUP don't exist on Windows; guard each installation
# with hasattr so ``python -m tui_gateway.entry`` (spawned by
# ``hermes --tui``) imports cleanly there. SIGBREAK (Windows' Ctrl+Break)
# is installed when available as a weaker equivalent of SIGHUP.
if hasattr(signal, "SIGPIPE"):
signal.signal(signal.SIGPIPE, signal.SIG_IGN)
if hasattr(signal, "SIGTERM"):
signal.signal(signal.SIGTERM, _log_signal)
if hasattr(signal, "SIGHUP"):
signal.signal(signal.SIGHUP, _log_signal)
elif hasattr(signal, "SIGBREAK"):
# Windows-only: Ctrl+Break in a console window delivers SIGBREAK.
# Route it through the same handler so kills are diagnosable.
signal.signal(signal.SIGBREAK, _log_signal)
if hasattr(signal, "SIGINT"):
signal.signal(signal.SIGINT, signal.SIG_IGN)
def _log_exit(reason: str) -> None:
+3 -3
View File
@@ -660,7 +660,7 @@ def _load_cfg() -> dict:
if _cfg_cache is not None and _cfg_mtime == mtime and _cfg_path == p:
return copy.deepcopy(_cfg_cache)
if p.exists():
with open(p) as f:
with open(p, encoding="utf-8") as f:
data = yaml.safe_load(f) or {}
else:
data = {}
@@ -679,7 +679,7 @@ def _save_cfg(cfg: dict):
import yaml
path = _hermes_home / "config.yaml"
with open(path, "w") as f:
with open(path, "w", encoding="utf-8") as f:
yaml.safe_dump(cfg, f)
with _cfg_lock:
_cfg_cache = copy.deepcopy(cfg)
@@ -2588,7 +2588,7 @@ def _(rid, params: dict) -> dict:
f"hermes_conversation_{_time.strftime('%Y%m%d_%H%M%S')}.json"
)
try:
with open(filename, "w") as f:
with open(filename, "w", encoding="utf-8") as f:
json.dump(
{
"model": getattr(session["agent"], "model", ""),
+11 -1
View File
@@ -95,7 +95,17 @@ pytest tests/ -v
## Cross-Platform Compatibility
Hermes officially supports Linux, macOS, and WSL2. Native Windows is **not supported**, but the codebase includes some defensive coding patterns to avoid hard crashes in edge cases. Key rules:
Hermes officially supports **Linux, macOS, WSL2, and native Windows** (via PowerShell install). Native Windows uses Git Bash (from [Git for Windows](https://git-scm.com/download/win)) for shell commands. A few features require POSIX kernel primitives and are gated: the dashboard's embedded PTY terminal pane (`/chat` tab) is WSL2-only.
When contributing code, keep these rules in mind:
- **Don't add unguarded `signal.SIGKILL` references.** It's not defined on Windows. Either route through `gateway.status.terminate_pid(pid, force=True)` (the centralized primitive that does `taskkill /T /F` on Windows and SIGKILL on POSIX), or fall back with `getattr(signal, "SIGKILL", signal.SIGTERM)`.
- **Catch `OSError` alongside `ProcessLookupError` on `os.kill(pid, 0)` probes.** Windows raises `OSError` (WinError 87, "parameter is incorrect") for an already-gone PID instead of `ProcessLookupError`.
- **Don't force the terminal to POSIX semantics.** `os.setsid`, `os.killpg`, `os.getpgid`, `os.fork` all raise on Windows — gate them with `if sys.platform != "win32":` or `if os.name != "nt":`.
- **Open files with an explicit `encoding="utf-8"`.** The Python default on Windows is the system locale (often cp1252), which mojibakes or crashes on non-Latin text.
- **Use `pathlib.Path` / `os.path.join` — never manually concat with `/`.** This matters less for strings the OS gives us back and more for strings we construct to hand to subprocesses.
Key patterns:
### 1. `termios` and `fcntl` are Unix-only
+32 -3
View File
@@ -1,7 +1,7 @@
---
sidebar_position: 2
title: "Installation"
description: "Install Hermes Agent on Linux, macOS, WSL2, or Android via Termux"
description: "Install Hermes Agent on Linux, macOS, WSL2, native Windows, or Android via Termux"
---
# Installation
@@ -16,6 +16,26 @@ Get Hermes Agent up and running in under two minutes with the one-line installer
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
```
### Windows (native, PowerShell)
Open PowerShell and run:
```powershell
irm https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.ps1 | iex
```
The installer handles **everything**: `uv`, Python 3.11, Node.js 22, `ripgrep`, `ffmpeg`, **and a portable Git Bash** (MinGit — a slim, self-contained Git for Windows distribution that Hermes uses for shell commands). It clones the repo under `%LOCALAPPDATA%\hermes\hermes-agent`, creates a virtualenv, and adds `hermes` to your **User PATH**. Restart your terminal (or open a new PowerShell window) after the install so PATH picks up.
**How Git is handled:**
1. If `git` is already on your PATH, the installer uses your existing install.
2. Otherwise it downloads portable **MinGit** (~45MB, from the official `git-for-windows` GitHub release) and unpacks it to `%LOCALAPPDATA%\hermes\git`. No admin rights required. Completely isolated — it won't interfere with any system Git install, broken or otherwise.
**Why not use winget?** Earlier designs auto-installed Git via `winget install Git.Git`, but winget fails badly when a system Git install is in a partial or broken state (exactly when users need the installer to just work). The portable MinGit approach sidesteps winget, the Windows installer registry, and any existing system Git entirely. If the Hermes Git install itself ever breaks, `Remove-Item %LOCALAPPDATA%\hermes\git` and re-run the installer — no system impact, no uninstall drama.
The installer also sets `HERMES_GIT_BASH_PATH` to the located `bash.exe` so Hermes resolves it deterministically in fresh shells.
If you prefer WSL2, the Linux installer above works inside it; both native and WSL installs can coexist without conflict (native data lives under `%LOCALAPPDATA%\hermes`, WSL data lives under `~/.hermes`).
### Android / Termux
Hermes now ships a Termux-aware installer path too:
@@ -33,8 +53,17 @@ The installer detects Termux automatically and switches to a tested Android flow
If you want the fully explicit path, follow the dedicated [Termux guide](./termux.md).
:::warning Windows
Native Windows is **not supported**. Please install [WSL2](https://learn.microsoft.com/en-us/windows/wsl/install) and run Hermes Agent from there. The install command above works inside WSL2.
:::note Windows Feature Parity
Everything except the browser-based dashboard chat terminal runs natively on Windows:
- **CLI (`hermes chat`, `hermes setup`, `hermes gateway`, …)** — native, uses your default terminal
- **Gateway (Telegram, Discord, Slack, …)** — native, runs as a background PowerShell process
- **Cron scheduler** — native
- **Browser tool** — native (Chromium via Node.js)
- **MCP servers** — native (stdio and HTTP transports both supported)
- **Dashboard `/chat` terminal pane****WSL2 only** (uses a POSIX PTY; native Windows has no equivalent). The rest of the dashboard (sessions, jobs, metrics) works natively — only the embedded PTY terminal tab is gated.
Set `HERMES_DISABLE_WINDOWS_UTF8=1` in your environment if you hit an encoding-related bug and want to fall back to the legacy cp1252 stdio path (useful for bisecting).
:::
### What the Installer Does
@@ -80,7 +80,7 @@ The **Chat** tab embeds the full Hermes TUI (the same interface you get from `he
- Node.js (same requirement as `hermes --tui`; the TUI bundle is built on first launch)
- `ptyprocess` — installed by the `pty` extra (`pip install 'hermes-agent[web,pty]'`, or `[all]` covers both)
- POSIX kernel (Linux, macOS, or WSL). Native Windows Python is not supported — use WSL.
- POSIX kernel (Linux, macOS, or WSL2). The `/chat` terminal pane specifically needs a POSIX PTY — native Windows Python has no equivalent, so on a native Windows install the rest of the dashboard (sessions, jobs, metrics, config editor) works but the `/chat` tab will show a banner telling you to use WSL2 for that feature.
Close the browser tab and the PTY is reaped cleanly on the server. Re-opening spawns a fresh session.
@@ -7,7 +7,18 @@ sidebar_position: 2
# Windows (WSL2) Guide
Hermes Agent is developed and tested on **Linux** and **macOS**. Native Windows is not supported — on Windows you run Hermes inside **WSL2** (Windows Subsystem for Linux, version 2). That means there are effectively two computers in play: your Windows host, and a Linux VM managed by WSL. Most confusion comes from not being sure which one you're on at any moment.
Hermes Agent now supports **both** native Windows and WSL2. This page covers the WSL2 path; for the native PowerShell install see [Installation](../getting-started/installation.md#windows-native-powershell).
**When to pick WSL2 over native:**
- You want to use the dashboard's embedded terminal (`/chat` tab) — that pane requires a POSIX PTY and is WSL2-only.
- You're doing POSIX-heavy development work and want your Hermes sessions to share the same filesystem / paths as your dev tools.
- You already have a WSL2 environment and don't want to maintain a second install.
**When native is fine (or better):**
- Interactive chat, gateway (Telegram/Discord/etc.), cron scheduler, browser tool, MCP servers, and most Hermes features all run natively on Windows.
- You don't want to think about crossing the WSL↔Windows boundary every time you reference a file or open a URL.
In WSL2 there are effectively two computers in play: your Windows host, and a Linux VM managed by WSL. Most confusion comes from not being sure which one you're on at any moment.
This guide covers the parts of that split that specifically affect Hermes: installing WSL2, getting files back and forth between Windows and Linux, networking in both directions, and the pitfalls people actually hit.
@@ -15,11 +26,13 @@ This guide covers the parts of that split that specifically affect Hermes: insta
A Chinese-language walkthrough of the minimum install path is maintained on this same page — switch via the **language** menu (top right) and select **简体中文**.
:::
## Why WSL2 (and not "just Windows")
## Why WSL2 (vs. native Windows)
Hermes assumes a POSIX environment: `fork`, `/tmp`, UNIX sockets, signal semantics, PTY-backed terminals, shells like `bash`/`zsh`, and tools like `rg`, `git`, `ffmpeg` that behave the way they do on Linux. Rewriting that for native Windows would be a full port — WSL2 gives you a real Linux kernel in a lightweight VM instead, and Hermes inside it is essentially identical to running on Ubuntu.
The native Windows install runs in Windows directly: your Windows terminal (PowerShell, Windows Terminal, etc.), Windows filesystem paths (`C:\Users\…`), and Windows processes. Hermes uses Git Bash to run shell commands, which is how Claude Code and other agents handle Windows today — it sidesteps the POSIX-vs-Windows gap without a full rewrite.
Practical consequences of this choice:
WSL2 runs a real Linux kernel in a lightweight VM, so Hermes inside it is essentially identical to running on Ubuntu. That's valuable when you want a real POSIX environment: `fork`, `/tmp`, UNIX sockets, signal semantics, PTY-backed terminals, shells like `bash`/`zsh`, and tools like `rg`, `git`, `ffmpeg` that behave the way they do on Linux.
Practical consequences of WSL2:
- The Hermes CLI, gateway, sessions, memory, skills, and tool runtimes all live inside the Linux VM.
- Windows programs (browsers, native apps, Chrome with your logged-in profile) live outside it.
+3 -3
View File
@@ -69,7 +69,7 @@ def extract_local_skills():
continue
skill_path = os.path.join(root, "SKILL.md")
with open(skill_path) as f:
with open(skill_path, encoding="utf-8") as f:
content = f.read()
if not content.startswith("---"):
@@ -128,7 +128,7 @@ def extract_cached_index_skills():
filepath = os.path.join(INDEX_CACHE_DIR, filename)
try:
with open(filepath) as f:
with open(filepath, encoding="utf-8") as f:
data = json.load(f)
except (json.JSONDecodeError, OSError):
continue
@@ -254,7 +254,7 @@ def main():
))
os.makedirs(os.path.dirname(OUTPUT), exist_ok=True)
with open(OUTPUT, "w") as f:
with open(OUTPUT, "w", encoding="utf-8") as f:
json.dump(all_skills, f, indent=2)
print(f"Extracted {len(all_skills)} skills to {OUTPUT}")